-- Logs begin at Thu 2022-07-28 16:46:57 UTC, end at Thu 2022-07-28 18:21:07 UTC. -- Jul 28 17:40:08.828186 dv-cinder-837543-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 17:40:08.840086 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 17:40:08 2022] *** Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: nodename: dv-cinder-837543-7-iscsi Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: machine: x86_64 Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: clock source: unix Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: pcre jit disabled Jul 28 17:40:08.842137 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: detected number of CPU cores: 4 Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: current working directory: / Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: detected binary path: /usr/bin/uwsgi-core Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: your processes number limit is 47680 Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: your memory page size is 4096 bytes Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: detected max file descriptor number: 2048 Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: lock engine: pthread robust mutexes Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: thunder lock: enabled Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 17:40:08.843405 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 17:40:08.882378 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: Python main interpreter initialized at 0x55ffabd669c0 Jul 28 17:40:08.882378 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: python threads support enabled Jul 28 17:40:08.882378 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: your server socket listen backlog is limited to 100 connections Jul 28 17:40:08.882378 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: your mercy for graceful operations on workers is 90 seconds Jul 28 17:40:08.882781 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 17:40:08.882858 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: *** Operational MODE: preforking *** Jul 28 17:40:08.883120 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: *** uWSGI is running in multiple interpreter mode *** Jul 28 17:40:08.883120 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: spawned uWSGI master process (pid: 92070) Jul 28 17:40:08.883334 dv-cinder-837543-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 17:40:08.884487 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: spawned uWSGI worker 1 (pid: 92083, cores: 1) Jul 28 17:40:08.884487 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: spawned uWSGI worker 2 (pid: 92084, cores: 1) Jul 28 17:40:08.884487 dv-cinder-837543-7-iscsi devstack@keystone.service[92070]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 17:40:14.599606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:40:14.599606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: from cryptography.utils import int_from_bytes Jul 28 17:40:14.600467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:40:14.600467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: from cryptography.utils import int_from_bytes Jul 28 17:40:14.600749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:40:14.600749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: from cryptography.utils import int_from_bytes Jul 28 17:40:14.601245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:40:14.601245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: from cryptography.utils import int_from_bytes Jul 28 17:40:14.651485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:40:14.651723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:40:14.651857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] command line args: [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:40:14.652124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:40:14.652303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ================================================================================ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:40:14.652568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] admin_token = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.652693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:40:14.652819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:40:14.652931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_dir = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.653054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] command line args: [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:40:14.653173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.653173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] config_source = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.653407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:40:14.653531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] debug = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.653921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ================================================================================ {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:40:14.654116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.656309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] admin_token = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.656309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] config_dir = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.656665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.656786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.656974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] config_source = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] debug = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.657912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_config_append = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_dir = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.658910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_config_append = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_dir = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.659948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_file = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.660996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.661972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.662883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] publish_errors = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.663984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.664413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.664538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.664644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] publish_errors = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.664765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] standard_threads = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.664880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_journal = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_json = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.665889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_stderr = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] standard_threads = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] use_syslog = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.666816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.666922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] use_journal = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.667113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.667219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] use_json = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.667219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] use_stderr = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.667442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.667573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] use_syslog = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.667679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.667801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:40:14.667916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.668937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.669970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.670702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.670822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.670937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671419 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.671985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.672943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.673963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.674904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.675923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.676970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677589 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.677907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.678986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.679855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.680893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.681985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.682980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.683904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684479 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.684964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.685953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.686918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.687977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.688943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.689951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.690903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.691906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.692812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.external = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.693915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.password = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] auth.token = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.694947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.external = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.695976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.password = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] auth.token = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.696945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.697912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.698991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.699921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.700927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.701962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.702966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.703925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.704992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.705953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.706954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.707895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.708956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.709890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.710955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.711916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.712993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.713941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.714949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.715901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.716848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.717456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.717572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.717933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.718961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.719971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.720981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.721867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.722951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.723956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] role.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.724928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] role.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.725929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.726968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.727928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.728957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.729951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.730963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.731899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.732065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.732144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.732308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.732395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.733976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.734999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.735979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.736863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.737867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.connection = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.738924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.739968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.740953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.741774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92083) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:40:14.741890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.742076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.742313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.742587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.742830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:40:14.743004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92084) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:40:14.772956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.773215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.773394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.773623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.773864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.774107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.783012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.783894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.814449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.814689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.815018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.815256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.870005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.872311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.923635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.923869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.927848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.928093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.991704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.991916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.992976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:14.993193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.016538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.016782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.017942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.018139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.018376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.030099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.031337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.043888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.044892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.045069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.045238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.045356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.045434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92084) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.045567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92083) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:40:15.063133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.063280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.063362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:40:15.063438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:40:15.063672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.063754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.063876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:40:15.063957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:40:15.064149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.064925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.065006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.074693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.074914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.074999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.075197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.075278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.075469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.075550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.075759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.082992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.083545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.087819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.088105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.088339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.088424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.088634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.088733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.088921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.089991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.090830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.091098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.114395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.114941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.116081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.116365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.116483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.116770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.121129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.121336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.121474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.121605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.121749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.121980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.129178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.129321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.129465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.129568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.133679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.133803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.133969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.134953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.135794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.152124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.152253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.152419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.152580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.156405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.156618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.159605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.159772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.159894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.159983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.164293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.164408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.167489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.167598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.170838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.171812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.172004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.172389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.172571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.172711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.189509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.189799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.190068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.190334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.192098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.192395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.192669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.192940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.197261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.198857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.199133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.199871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.201965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.202247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.202906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.203191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.203492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.203780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.204075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.205111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.205396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.205700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.205983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.206271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.215612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.215896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.216181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.216467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.216750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.217412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.217711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.217907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.218201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.218512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.218795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.219077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.219352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.219630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.233416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.235392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.236427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.238414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.239400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.241358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.242003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.243584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.243871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.244030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.244228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.245535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.245820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.246109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.251669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.253133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.253307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.254765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.256275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.256575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.256860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.261415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.261649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.261649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.261649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.261649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.265966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.266259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.266578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.268340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.268645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.268923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.273454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92083) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.273737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.274021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.274300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.274605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.274895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.275167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.275457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.275801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92084) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:40:15.275950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.276996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.277145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.277284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92083) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.277410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.277680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.278052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.278342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.278646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.278931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.279199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.279876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92084) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:40:15.311599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ffabd669c0 pid: 92083 (default app) Jul 28 17:40:15.314326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ffabd669c0 pid: 92084 (default app) Jul 28 17:40:15.315102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4265-de6f-4848-bfe0-c0bc960b76d0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:15.315402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4265-de6f-4848-bfe0-c0bc960b76d0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:15.315682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4265-de6f-4848-bfe0-c0bc960b76d0 None None] PATH_INFO: `/v3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:15.320020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Thu Jul 28 17:40:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:23.934631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:23.935241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:23.935241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:23.940260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1/2] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:23.945613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:23.946041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:23.946182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:24.019047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: INFO dbcounter [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] Registered counter for database keystone Jul 28 17:40:24.040822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG oslo_db.sqlalchemy.engines [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92083) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:40:24.047516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writer thread running {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:40:24.225095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG passlib.handlers.bcrypt [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92083) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:40:24.225435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG passlib.handlers.bcrypt [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92083) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:40:24.280217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:24.281812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:24.407576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 2/3] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 17:40:24.563170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:24.632902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: INFO dbcounter [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] Registered counter for database keystone Jul 28 17:40:24.651957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG oslo_db.sqlalchemy.engines [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92084) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:40:24.658294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writer thread running {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:40:24.778058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74f261-5d78-4037-a97d-458ce8b4fdc7 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:24.782872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:24.783506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:24.783787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:24.784060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:24.792501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:24.846297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:40:24.846677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2c8f9-0a61-4fad-8a49-796a82d13426 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:24.858280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 2/4] 10.222.0.85 () {58 vars in 1324 bytes} [Thu Jul 28 17:40:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:40:24.864554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:24.880046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e08da9-6acc-4745-93da-d7a49a5985f7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:24.884591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:24.885732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:24.886030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:24.886320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] PATH_INFO: `/v3/projects/eaffcb498d9c405ca0d18c97b43881f2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:24.887868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] RBAC: Authorizing `identity:get_project(project_id=eaffcb498d9c405ca0d18c97b43881f2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:24.940031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING keystone.common.rbac_enforcer.enforcer [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:40:24.940372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399200-8d59-4e4d-a24c-8afbfd35a94f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:24.945049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 3/5] 10.222.0.85 () {56 vars in 1207 bytes} [Thu Jul 28 17:40:24 2022] GET /identity/v3/projects/eaffcb498d9c405ca0d18c97b43881f2 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:26.911862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e8b4-92c7-4346-8411-bac116544d33 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:26.912363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e8b4-92c7-4346-8411-bac116544d33 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:26.912465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e8b4-92c7-4346-8411-bac116544d33 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:26.916488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 3/6] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:26.922988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:26.923377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:26.923551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:26.944092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:26.945630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:26.953628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 4/7] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 17:40:26.975515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e8b4-92c7-4346-8411-bac116544d33 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:26.988650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e8b4-92c7-4346-8411-bac116544d33 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:26.992900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:26.993499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:26.993783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:26.994061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:27.002746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:27.004517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd5327-579e-4575-a377-7a47f87b9ff2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:27.013299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 4/8] 10.222.0.85 () {58 vars in 1324 bytes} [Thu Jul 28 17:40:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:40:27.018965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:27.028122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f436775-53b4-49a5-92b2-283b7ffe9a9d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:27.032854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:27.033499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:27.033788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:27.034074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:27.035493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:27.037341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e604b4-e7a2-4856-b687-e0ab9fe52afc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:27.039006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 5/9] 10.222.0.85 () {56 vars in 1198 bytes} [Thu Jul 28 17:40:27 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:29.103033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a2128-b45b-421b-9a3d-5b566af661ac None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.103548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a2128-b45b-421b-9a3d-5b566af661ac None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.103548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a2128-b45b-421b-9a3d-5b566af661ac None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.104844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 5/10] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:29.110535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.110939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.111109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.131458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:29.133129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475220bc-c5fb-41f3-a833-b949d7825f9b None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.133296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:29.133582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475220bc-c5fb-41f3-a833-b949d7825f9b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.134029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475220bc-c5fb-41f3-a833-b949d7825f9b None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.136747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 6/11] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:29.142515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.142980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.143440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.212281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG passlib.handlers.bcrypt [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92084) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:40:29.212966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG passlib.handlers.bcrypt [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92084) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:40:29.250618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 6/12] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:29.262395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:29.264560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:29.274103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.299356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.366931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 7/13] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:40:29.383930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30e07-aeea-4328-9a68-392457fd6cf0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:29.389835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.420136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:29.420825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.421113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.421394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.432784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:29.442328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:29.443314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:29.444206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 7/14] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:40:29.449244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.495519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64497421-54c8-4727-b096-a56e4f29a5cc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:29.498441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.531975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.534210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:29.535129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.535465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.535774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.550153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fac2-5d72-43ea-a495-3cf51db9787a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:29.586169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:29.586893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.587179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.587453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.588174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:29.588484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:29.589917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:29.601965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 8/15] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:29.606370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:29.607704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.608180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:29.711127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 8/16] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:40:29.713591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cea4a1-a2b2-4b4d-852a-63448c32cf00 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:29.716955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.719780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.827051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8014a-c3c9-4aee-88f1-619cd6384f65 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:29.861369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:29.862016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.862299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.862607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.863767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:29.864817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:29.865227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:29.865710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:29.865994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:29.866276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:29.866884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 9/17] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:29.873059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.885013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:29.886527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:29.887967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:40:29.906238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:29.907129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:29.908053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:40:29.908053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:40:29.908053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:40:29.908053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:29.908053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 9/18] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:40:29.913266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:29.970953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:29.975879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab110601-623e-4d84-ab3a-56251acd607b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:30.010883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:30.011537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:30.011824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:30.012111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:30.013800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:30.015223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f700-2661-4a52-bf0a-b87d6d1c63f4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:30.015335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:30.016884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 10/19] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:30.022882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:30.050371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:30.050989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:30.051276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:30.051618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:30.052276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:30.052586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:30.053980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea272c-8a0e-446a-903f-8d553a19b800 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:30.071609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 10/20] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:30.108492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:30.131203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ffaf1-f931-45c8-8390-0682786079d7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:30.170960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:30.171602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:30.172144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:30.172423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:30.173235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.domain.id=default, user.id=5580a3ecfd9d47fb8a889209b62e0089) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:30.173549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:30.175053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e0a0a-79e1-42d5-b694-0f399b0c1439 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:30.194393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 11/21] 10.222.0.85 () {56 vars in 1322 bytes} [Thu Jul 28 17:40:30 2022] GET /identity/v3/role_assignments?user.id=5580a3ecfd9d47fb8a889209b62e0089&scope.domain.id=default&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:32.387226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d626517-bc8a-4fac-a736-5b38db40d97d None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:32.388743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d626517-bc8a-4fac-a736-5b38db40d97d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:32.389218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d626517-bc8a-4fac-a736-5b38db40d97d None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:32.390819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 11/22] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:32.396327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b96a-34de-45ca-bbcb-03b394b15b83 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:32.397134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b96a-34de-45ca-bbcb-03b394b15b83 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:32.397615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b96a-34de-45ca-bbcb-03b394b15b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:32.423579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6108b96a-34de-45ca-bbcb-03b394b15b83 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:32.424798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6108b96a-34de-45ca-bbcb-03b394b15b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:32.522123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 12/23] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:32.557276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d626517-bc8a-4fac-a736-5b38db40d97d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:32.666120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe531-2e40-45ad-809c-234688a04edb None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:32.666331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe531-2e40-45ad-809c-234688a04edb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:32.666869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbe531-2e40-45ad-809c-234688a04edb None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:32.668414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 13/24] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:32.679196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c559-3689-4d88-af25-030aac95c31a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:32.679777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c559-3689-4d88-af25-030aac95c31a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:32.680683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815c559-3689-4d88-af25-030aac95c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:32.715642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c815c559-3689-4d88-af25-030aac95c31a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:32.718436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c815c559-3689-4d88-af25-030aac95c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:32.748090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d626517-bc8a-4fac-a736-5b38db40d97d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:32.782211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:32.840779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:32.842119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:32.842606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:32.842890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:32.863733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:32.865570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:32.892492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 14/25] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:40:32.898536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:32.899578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:32.900403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:40:32.900403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:40:32.900403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:40:32.900403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:32.900856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 12/26] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:32.907220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815c559-3689-4d88-af25-030aac95c31a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:32.927039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:32.932170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:33.094391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815c559-3689-4d88-af25-030aac95c31a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:33.146376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3be65-c821-413f-bc62-ea8f4bfb35c0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:33.157636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:33.167335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.167860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.168338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.169904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:33.170554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:33.172481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b18b43-8ff9-4a11-9c50-b530300b9c8d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:33.191994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 15/27] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:33.193757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954999a8-a023-46b6-a408-5a18a80eaca5 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.194268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954999a8-a023-46b6-a408-5a18a80eaca5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.194809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954999a8-a023-46b6-a408-5a18a80eaca5 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.196638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 16/28] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:33.201193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:33.201868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.202154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.202455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.203304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa40-8a00-4a96-a1c1-a38bc353467e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.203799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa40-8a00-4a96-a1c1-a38bc353467e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.204277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa40-8a00-4a96-a1c1-a38bc353467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.236570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:33.248118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c79faa40-8a00-4a96-a1c1-a38bc353467e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:33.249690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c79faa40-8a00-4a96-a1c1-a38bc353467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:33.393790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:33.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:33.396033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:33.397553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:33.400517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 13/29] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:33.401737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:33.415694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 17/30] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:33.416980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ff82-4e68-4fd5-9770-bc6927fac516 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.417271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ff82-4e68-4fd5-9770-bc6927fac516 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.417600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ff82-4e68-4fd5-9770-bc6927fac516 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.418884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 18/31] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:33.420252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ff82-4e68-4fd5-9770-bc6927fac516 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:33.440903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:33.530291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b3a65-c1c4-4667-bab2-a7fe37a4f6d7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:33.568005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:33.568645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.568991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.569330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.570249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:33.571784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff182-3dfe-4eaa-8e62-e84e3dafb47d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:33.574866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ff82-4e68-4fd5-9770-bc6927fac516 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:33.615720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:33.616346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.616634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.616912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.617638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:33.617948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:33.619736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766704d-b7ca-4783-a18a-ea3627b26c24 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:33.630379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 19/32] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:33.631718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.632017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.632302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.653392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:33.655156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:33.709366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:33.725953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 20/33] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:33.734504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:33.853071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 14/34] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:40:33 2022] POST /identity/v3/projects => generated 301 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:33.854659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ad59-62dd-4971-87ad-12a75276e6b8 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.854957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ad59-62dd-4971-87ad-12a75276e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.855245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ad59-62dd-4971-87ad-12a75276e6b8 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.858392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 15/35] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:33.858392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ad59-62dd-4971-87ad-12a75276e6b8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:33.866735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae828a62-e90d-4b5a-b7f9-ffffc6060945 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:33.921270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:33.922928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:33.923366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:33.924090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:33.924986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:33.926645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb673f8-ec3a-4d50-a871-51d0e3928182 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.011508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:34.039321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ad59-62dd-4971-87ad-12a75276e6b8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.086947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.087614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.087904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.088198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.089826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.091758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.094813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 16/36] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:33 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:34.095373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.162102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 21/37] 10.222.0.85 () {60 vars in 1131 bytes} [Thu Jul 28 17:40:33 2022] POST /identity/v3/roles => generated 219 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:34.163712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.164013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.164327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.185570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:34.187306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:34.218775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b86c-be99-420c-969f-533b3343ea69 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.240091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:34.261322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.261967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.262292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.262623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.264162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.265782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf53b33-8a47-4fb4-abe6-2a1affe3b5dd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.268716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 22/38] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:34.275356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.346816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 17/39] 10.222.0.85 () {60 vars in 1131 bytes} [Thu Jul 28 17:40:34 2022] POST /identity/v3/roles => generated 225 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:34.348059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ed8c3-d071-46b3-8cb3-dbc13d355b56 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.348355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ed8c3-d071-46b3-8cb3-dbc13d355b56 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.348644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ed8c3-d071-46b3-8cb3-dbc13d355b56 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.351659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 18/40] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:34.352535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ed8c3-d071-46b3-8cb3-dbc13d355b56 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.381812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:34.405278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c824-8a43-4bd8-8ef0-f15b4a0fc223 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.450914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.451632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.451882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.452201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.453539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.455195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277a4f3-58f3-4de8-955f-79ad3dc5868a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.461701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 23/41] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:34.461701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.461701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.461701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.480472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:34.481747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:34.507554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ed8c3-d071-46b3-8cb3-dbc13d355b56 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.532621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:34.551345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.552081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.552370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.552648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.553373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.554834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.569021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 24/42] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:34.572595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.623350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 19/43] 10.222.0.85 () {60 vars in 1131 bytes} [Thu Jul 28 17:40:34 2022] POST /identity/v3/roles => generated 223 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:34.624249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.686615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bce97c-45fe-4415-be75-a149efac9568 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.708549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:34.728916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.729927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.730124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.730385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] PATH_INFO: `/v3/domains/default/users/5580a3ecfd9d47fb8a889209b62e0089/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.732896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5580a3ecfd9d47fb8a889209b62e0089, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.734473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff92921-7d84-4d82-a883-95db1dc28b1b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.748179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be541e-f644-4b12-80f5-808c04263562 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.786083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.786935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.787158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.787436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.788735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.790252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.792053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 20/44] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:34.798332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:34.804054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 25/45] 10.222.0.85 () {58 vars in 1361 bytes} [Thu Jul 28 17:40:34 2022] PUT /identity/v3/domains/default/users/5580a3ecfd9d47fb8a889209b62e0089/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:34.921038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c393-091e-4526-95c1-1f9c16b9b378 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:34.958474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:34.959121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:34.959415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:34.959702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:34.960623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:34.962186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e6c90-70e3-418d-b3d4-629fd002c773 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:34.968422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:35.053088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 21/46] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:40:34 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:40:36.808857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb93-28a3-4a0a-b052-7015a3a16d2e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:36.809379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb93-28a3-4a0a-b052-7015a3a16d2e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:36.809462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb93-28a3-4a0a-b052-7015a3a16d2e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:36.810905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 26/47] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:36.816028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd8e4f-588c-48dd-967f-81961c29eca3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:36.816432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd8e4f-588c-48dd-967f-81961c29eca3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:36.816590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd8e4f-588c-48dd-967f-81961c29eca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:36.836025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a7cd8e4f-588c-48dd-967f-81961c29eca3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:36.837098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a7cd8e4f-588c-48dd-967f-81961c29eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:36.906470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 22/48] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:36.931460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dabb93-28a3-4a0a-b052-7015a3a16d2e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:36.969010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.040602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dabb93-28a3-4a0a-b052-7015a3a16d2e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.077237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.077806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.078095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.078400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.088600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:37.093586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:37.094591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:37.095524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 27/49] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:37.098896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51810d-1809-43f9-9288-8cedae55ce97 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.099197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51810d-1809-43f9-9288-8cedae55ce97 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.099487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51810d-1809-43f9-9288-8cedae55ce97 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.100427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.100837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 23/50] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:37.105585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.105877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.106157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.125657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:37.126895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.196647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 24/51] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:40:37.198666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.206123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0d4a9-c074-4aec-bb49-8edb1568c8ec None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.220690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.241889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.242564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.242847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.243167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.243985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:37.247870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.247870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.259773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 28/52] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:37.265878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.275575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.330838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892e831-4ea2-466c-8819-5461d354b4b5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.367212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.367868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.368155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.368431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.371327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.371327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.371846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3526-f852-4dd8-9c25-9f320e4af394 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.373499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 25/53] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:37.379237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.408269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.408928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.409219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.409503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.410853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.412334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.414073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 29/54] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:37.420135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.446267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.483513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c65f46-364d-4dce-8031-3a1bbe622beb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.501601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.520078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.520703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.520993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.521273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.521990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.522214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e7660-ebc5-42fc-977d-e59431309d0d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.523522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80337873-c055-4e47-957e-651004c35ba3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.557366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.558016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.558341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.558658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.560600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.562067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.563773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 30/55] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:37.569490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.635657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 26/56] 10.222.0.85 () {60 vars in 1140 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/projects => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:40:37.673732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.684040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e842ef-5880-4d56-a831-e9c97d303392 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.726215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.726907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.727199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.727486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.728306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.domain.id=default, user.id=5580a3ecfd9d47fb8a889209b62e0089) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:37.728626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.730155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830ea84-f7cb-42ce-9bb9-01125e6659d9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.737206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 31/57] 10.222.0.85 () {56 vars in 1322 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/role_assignments?user.id=5580a3ecfd9d47fb8a889209b62e0089&scope.domain.id=default&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:39.681107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b491cf-5f42-494f-a078-305ea68f4421 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.681621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b491cf-5f42-494f-a078-305ea68f4421 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.681704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b491cf-5f42-494f-a078-305ea68f4421 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.683071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 27/58] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:39.688324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.688765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.688928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.707714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:39.708935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.776678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 32/59] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:39.801745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b491cf-5f42-494f-a078-305ea68f4421 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:39.918869 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b491cf-5f42-494f-a078-305ea68f4421 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:39.955484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:39.956052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.956341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.956624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.957787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:39.959294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad647c9-1843-4a44-b66c-1681b8e8f852 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:39.960868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 28/60] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:39.966626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:40.051839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:40.067015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909394d-3ab5-47ca-b750-b95d0a24e83a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:40.100494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:40.101109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:40.101392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:40.101668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:40.102380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:40.103833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac60086-bba5-48c5-8287-ab185758275c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:40.192663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 33/61] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:40:39 2022] POST /identity/v3/users => generated 275 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:43.557571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de99a-88d5-473b-a2cc-d424973fea17 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.558629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de99a-88d5-473b-a2cc-d424973fea17 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.558734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e70f2-5149-4cef-8c3e-b20844da63da None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.559076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e70f2-5149-4cef-8c3e-b20844da63da None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.559359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e70f2-5149-4cef-8c3e-b20844da63da None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.561438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75de99a-88d5-473b-a2cc-d424973fea17 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.561631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 34/62] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:43.562398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 29/63] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:43.566494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7455-6893-4497-9b5b-ccc4ba6234ec None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.567045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7455-6893-4497-9b5b-ccc4ba6234ec None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.567539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f7455-6893-4497-9b5b-ccc4ba6234ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.567727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.568025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.568318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.588557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a49f7455-6893-4497-9b5b-ccc4ba6234ec None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:43.590151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a49f7455-6893-4497-9b5b-ccc4ba6234ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:43.601128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:43.602369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:43.660528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:43.667002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 35/64] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:43.672861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722593-34d8-45f0-b97f-f38757cce375 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.673554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722593-34d8-45f0-b97f-f38757cce375 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.680268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1722593-34d8-45f0-b97f-f38757cce375 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.680268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 36/65] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:43.681230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.681726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.682121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.702440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 30/66] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:40:43.719060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:43.719234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:43.720682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:43.796093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 37/67] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:43.797550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:43.800237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:43.836337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea831-1af2-4ee7-9c7e-1877c110ebc1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:43.875363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:43.876268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.876732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.877156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.890575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:43.896428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:43.897410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:43.898314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:43.900742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 31/68] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:40:43.901776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:43.915813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b031727-938b-4ebb-9c75-7bfac413ecec None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:43.933736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:43.958448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673272f-6084-4843-bda5-25076708c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:43.959440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673272f-6084-4843-bda5-25076708c20d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:43.959874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673272f-6084-4843-bda5-25076708c20d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:43.960329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673272f-6084-4843-bda5-25076708c20d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:43.973261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673272f-6084-4843-bda5-25076708c20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f673272f-6084-4843-bda5-25076708c20d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:43.978997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:43.980073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:43.980980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:43.983639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 38/69] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:40:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:43.984604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673272f-6084-4843-bda5-25076708c20d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.022469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fde10-53b2-41fd-97a9-a7d563c372b6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.055469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.061704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.062576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.062914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.063261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.074212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:44.079731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:44.080587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:44.081345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 32/70] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:44.081969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.104508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673272f-6084-4843-bda5-25076708c20d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.144374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.145003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.145294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.145578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.146315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:44.146651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.148066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52174f-4aef-4c27-bdf0-383fb3fc4af0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.157534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 39/71] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.158838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236d2a-1804-4664-b018-d51c759777d2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.159125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236d2a-1804-4664-b018-d51c759777d2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.159411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236d2a-1804-4664-b018-d51c759777d2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.164706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 40/72] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:44.164706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5236d2a-1804-4664-b018-d51c759777d2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.182224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.198078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea8e0-a842-4106-88a6-8eeab005512c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.232954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.233594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.233884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.234169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.234949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:44.235263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.236658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.248683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 33/73] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.249523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.269925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5236d2a-1804-4664-b018-d51c759777d2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.294928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.305759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.311929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e205b-4102-463d-8583-ef78ec35c594 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.318341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 41/74] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.319703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.319996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.320275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.338878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:44.340096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:44.355998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572f700-e032-4c6a-aafb-f3e14739ee8a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.390592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.391236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.391526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.391806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.392999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.394475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e42d15-673b-4201-8ca6-a37d8ac83acf None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.396259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 34/75] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.397366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d2a3-88a8-4c80-890b-19f6aaa8c960 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.397658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d2a3-88a8-4c80-890b-19f6aaa8c960 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.397940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d2a3-88a8-4c80-890b-19f6aaa8c960 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.399165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 35/76] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:44.399865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d2a3-88a8-4c80-890b-19f6aaa8c960 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.416374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 42/77] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:44.417108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.424423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.506872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d2a3-88a8-4c80-890b-19f6aaa8c960 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.522213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e546c7-db33-4462-b24a-c020fc52361f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.539307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.542038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.542717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.543012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.543292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.557952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.558675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.558923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.559497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.560227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.560602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267c91-3ec7-43f9-8361-5b67061bdff6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.561849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.572414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 43/79] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.572579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 36/79] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:44.572654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.572750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.572750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.572750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.584747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:44.585965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:44.657200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 37/80] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:44.657986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.666256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6997a1-0c33-412b-83c8-a931dd240ce1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.671787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.701922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.702579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.702869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.703152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.756174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.771981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:44.773506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:44.776574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384e21-af58-47a4-8194-ce8b603dbdfe None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.810313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.810989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.811278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.811563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.821752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:44.827441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:44.828326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:44.829216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 38/81] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:44.832228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.880900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 44/82] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:44.882114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:44.899525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.939654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c19db-5b07-4f81-9c37-14a5c5aa7a6f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:44.970301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:44.979162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:44.979796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:44.982506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:44.982506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:44.987923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bd296-524e-44c2-902d-ba69a8ded962 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.023726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.024352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.024647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.024931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.049209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.050732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.099192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.100687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.112877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.153779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 39/83] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:45.154795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.185740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.203038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 45/84] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:45.205021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.256480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547692bb-f33f-4b30-b188-24767fbf9901 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.291082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.291728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.292018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.292294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.293458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.295022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.297066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 40/85] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.297666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.309655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044aa0-8994-4b9f-acbe-be7e90c07bde None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.331002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.344057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.344685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.344974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.345299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.345981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:45.346414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.347745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.356436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 46/86] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.357290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.404695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.405230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473469d0-f73b-457a-be66-59200c4ecd96 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.440328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.440972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.441266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.441548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.442723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.444178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.445826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 41/87] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.450838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.460346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5929d7-37c9-4bd6-be38-4bdfe8f74887 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.494856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.495499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.495785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.496064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.497621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.499100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.500776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 47/88] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.501612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.546765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.549745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c04dc2-31e6-4095-a626-472441fcccfb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.584039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.584691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.584982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.585258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.587082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.589512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.592400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 42/89] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.593577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.608358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9238-7373-4a34-bf6e-39196b290d24 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.621773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.643767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.644394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.644687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.644967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.645660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.647167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.693927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6f250-2446-4e95-8b6e-ddce2712d282 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.728271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.728851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.729176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.729491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.730749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.732523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.734404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 43/90] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.750380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 48/91] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:40:45 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:45.750380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.750704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.784044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:45.863775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02581313-3c58-43b5-9c03-14875ca22166 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.875563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bcbbb-bfae-438c-aa60-8938010c1632 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:45.906736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.907405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.907764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.907971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.908783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=5580a3ecfd9d47fb8a889209b62e0089) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:45.909103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.910657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.920197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:45.920545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 44/92] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/role_assignments?user.id=5580a3ecfd9d47fb8a889209b62e0089&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.921055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.921932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:45.922236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:45.922553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:45.923428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:45.923752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:45.926553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:45.942440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 49/93] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:45.942440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:45.988398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:46.056319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3177dd-b60f-4e97-a5e5-2f9873c40b4c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:46.058281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:46.084553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338ac4-bd0f-430b-86ad-14815e8a506b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:46.105173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:46.105759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:46.106737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:46.107693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:46.108543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:46.108870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:46.110744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:46.116780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 45/94] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:46.131686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:46.132350 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:46.132645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:46.132934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:46.275165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:46.276613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:46.320582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:46.414659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 50/95] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:46.417018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:46.562533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0135-0f01-4e52-81c8-8c81fb34e53b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:46.602073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:46.603546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:46.604038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:46.604520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] PATH_INFO: `/v3/projects/ce7940ca188e4a71a0b113e81dfddf85` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:46.606504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] RBAC: Authorizing `identity:get_project(project_id=ce7940ca188e4a71a0b113e81dfddf85)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:46.608287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a66b18-eb7e-4102-8110-b7c2ab7fec5a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:46.610215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 46/96] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:46 2022] GET /identity/v3/projects/ce7940ca188e4a71a0b113e81dfddf85 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:46.616303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:46.756287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5a940-7ab0-419e-b5ad-f4777951070f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:46.801444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:46.802491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:46.802986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:46.803471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:46.804477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=ce7940ca188e4a71a0b113e81dfddf85, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:46.805013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:46.806725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77e26-2e03-4e94-90e4-2026c7cafc6b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:46.809020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:46.814177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 51/97] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:46 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=ce7940ca188e4a71a0b113e81dfddf85&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:48.611821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd5fa-7dfb-4d32-87d0-4cf632e9733e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:48.612322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd5fa-7dfb-4d32-87d0-4cf632e9733e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:48.612425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd5fa-7dfb-4d32-87d0-4cf632e9733e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:48.613839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 47/98] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:48.619232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bca48-c988-4828-aaab-3618c1c9ae1e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:48.619716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bca48-c988-4828-aaab-3618c1c9ae1e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:48.619792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bca48-c988-4828-aaab-3618c1c9ae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:48.647498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-866bca48-c988-4828-aaab-3618c1c9ae1e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:48.648747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-866bca48-c988-4828-aaab-3618c1c9ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:48.740260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 52/99] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:48.775915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fd5fa-7dfb-4d32-87d0-4cf632e9733e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:48.812370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:48.918091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fd5fa-7dfb-4d32-87d0-4cf632e9733e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:48.974116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:48.974892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:48.975188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:48.975476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:48.995820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:48.999788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29309c-66fa-4066-8a0c-76bd4ec37ab9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.000084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29309c-66fa-4066-8a0c-76bd4ec37ab9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.000372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29309c-66fa-4066-8a0c-76bd4ec37ab9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.001832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 53/100] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:49.007019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94e21-227f-4694-8e07-8fc05efac0d2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.007313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94e21-227f-4694-8e07-8fc05efac0d2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.007601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94e21-227f-4694-8e07-8fc05efac0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:49.014596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:49.015511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:49.016277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:49.023937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 48/101] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:49.025814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.052266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f3b94e21-227f-4694-8e07-8fc05efac0d2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:49.053505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f3b94e21-227f-4694-8e07-8fc05efac0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:49.208196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.221668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798284fb-e1d4-4705-a7e3-ead9c877084b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.227232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 54/102] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:49.230231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb30d-f4e1-4532-8357-d003a9a04752 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.231396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb30d-f4e1-4532-8357-d003a9a04752 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.231920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb30d-f4e1-4532-8357-d003a9a04752 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.234337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 55/103] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:49.238889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad30f5-8526-40bc-af55-b7980e57c430 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.239426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad30f5-8526-40bc-af55-b7980e57c430 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad30f5-8526-40bc-af55-b7980e57c430 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 56/104] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:49.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.266243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:49.267492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:49.330446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.331348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.332029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.332526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.333464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:49.333975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:49.335587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccbd26-35e4-4b4e-978b-82516def114c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:49.345289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 49/105] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:49.350163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.351019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.351464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.351973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.366057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 57/106] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:49.372814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.373687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:49.375112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:49.454371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 50/107] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:49.460282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.499893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7125f-d19e-4166-b683-2e41948ad2cd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.538481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.539175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.539476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.539761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.551264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:49.556185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:49.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:49.557152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:49.558080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 58/108] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:49.559854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.575758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784252e9-70f6-4f30-9d6c-c7552230b8a3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.614461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.614650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.615270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.615580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.615861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.617074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:49.618590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e89761-e731-4dc9-9d80-bb0a61d8d13e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:49.620221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 51/109] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:49.621445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921c20-cf88-4a4e-b7eb-99d52370aa28 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.622315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921c20-cf88-4a4e-b7eb-99d52370aa28 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.622661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921c20-cf88-4a4e-b7eb-99d52370aa28 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.624198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 52/110] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:49.624777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921c20-cf88-4a4e-b7eb-99d52370aa28 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.685322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde180e7-f3b1-482f-ace8-64983be0943a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.726337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.726937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.727226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.727505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.738772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921c20-cf88-4a4e-b7eb-99d52370aa28 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.739231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:49.744213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:49.745053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:49.745800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 59/111] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:49.747594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.778527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.779527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.779876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.780164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.792636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:49.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:49.799724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:49.800906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 53/112] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:49.803185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:49.821563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.862715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1557209-77c6-44e6-85f1-5ddb6afe01c2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.872684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:49.898026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.898701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.898994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.899292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.900026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:49.900343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:49.901749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82036c-078d-4ca2-bba2-008025a98aab None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:49.911535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 60/113] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:49.913494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.914092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.914586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.933459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:49.934800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:49.937826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88709b0f-b9dd-4d86-8677-24a7eec1869c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:49.972366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:49.973055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:49.973348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:49.973631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:49.974852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:49.976336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:49.981570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 54/114] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:49.981570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.007393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 61/115] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:50.008309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.067786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.088309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bd34d-56d1-425c-bc7b-9c347398fafb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.115356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.117502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc7c39-4f88-4249-984c-fc14e2a8fde0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.123127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.123752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.124039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.124318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.125036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:50.125346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.126748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.134973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 55/116] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.135674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.154774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.155405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.155696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.155979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.156710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:50.157025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.158460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1dad-cfb2-4077-bc15-422721f9753b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.166554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 62/117] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.167925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabef46-f9b5-4a43-bad7-8ecab4779b2e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.168219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabef46-f9b5-4a43-bad7-8ecab4779b2e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.169163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabef46-f9b5-4a43-bad7-8ecab4779b2e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.170755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 63/118] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:50.171719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabef46-f9b5-4a43-bad7-8ecab4779b2e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.245405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd63a32-40ff-4dfd-81f5-63cff7c1b0bd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.277707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabef46-f9b5-4a43-bad7-8ecab4779b2e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.281168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.281821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.282119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.282474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] PATH_INFO: `/v3/users/52a0bdcaee774dfba031cea6614afff3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.294596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.297588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] RBAC: Authorizing `identity:get_user(user_id=52a0bdcaee774dfba031cea6614afff3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.299182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.303678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 56/119] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/users/52a0bdcaee774dfba031cea6614afff3 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.303678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.318554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.319235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.319527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.319811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/users/5580a3ecfd9d47fb8a889209b62e0089/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.321817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, user_id=5580a3ecfd9d47fb8a889209b62e0089, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.323321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.400301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25d5b-1575-420c-ad34-abded4cfa90f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.410367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 64/120] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:40:50 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/users/5580a3ecfd9d47fb8a889209b62e0089/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:50.430619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.440971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.441519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.442201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.442559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.442849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.444162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.445736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.447659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 57/121] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.448300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.526913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.533046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ce607-3f59-49be-8fce-fa4585c08d72 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.566936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd28413-c0dc-448b-a8fa-013250b2817c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.576719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.578087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.578087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.578278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.579456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.581050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb506d5b-1cd4-4b36-8478-c0766acd70b2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.582750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 65/122] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.583910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.584306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.584597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.604849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:50.606651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:50.612807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.613720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.614194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.614708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.616100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.617555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.619364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 58/123] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:50.619979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.682873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 66/124] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:50.683787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:50.713857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.747209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938ac36-a67c-4b27-b689-44d795aa75a4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.785116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.786085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.786644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.787033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.791529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.810946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22692039-0e7c-4549-9613-271e335d4a8d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:50.857333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:50.857972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:50.858260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:50.858583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:50.859271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.860732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.870853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:50.873200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:50.972780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 67/125] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:40:50 2022] POST /identity/v3/projects => generated 302 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:50.972780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.002042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 59/126] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:51.002851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.045160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.052059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.081557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593dc03-80b5-49e5-990a-6e5fc934d380 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.127233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.127913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.128198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.128479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.130021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.131868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.133847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 68/127] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:50 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:51.134561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.154166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813ebe0-f81e-403c-b884-b7758a45bf9a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.209557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.210757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.211457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.211752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.213005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.215081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.218035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 60/128] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:51.219326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.270503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802d81a-0097-4327-82e0-3da6225be8c1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.313730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.314298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.314636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.314919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.333069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:51.339818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:51.342737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:51.343641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:51.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:51.347168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 69/129] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:51.348535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.349791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4981b-ec67-4032-8f82-4ef841fcda3f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.353473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.389539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.390378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.390775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.391159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.392723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.394261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.401721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 61/130] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:51.403639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.484068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741f3c3-7092-4c45-93c2-4dcbc1e507fc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.526073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.526820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.527114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.527403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/users/59c9455e874f42308d26ae107456be33/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.529979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ea79-418f-4233-bd36-af4cce61b5f5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.530300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, user_id=59c9455e874f42308d26ae107456be33, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.531918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.571661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.572682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.573133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.573569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/users/59c9455e874f42308d26ae107456be33/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.577185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, user_id=59c9455e874f42308d26ae107456be33, role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.579083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.622705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 70/131] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:40:51 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/users/59c9455e874f42308d26ae107456be33/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:51.631426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.651664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 62/132] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:40:51 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/users/59c9455e874f42308d26ae107456be33/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:51.653605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:51.739002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.785024 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:51.856769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bc071-1156-41a1-93db-c519e9b4750e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.888692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994570-b9ec-49f4-807d-c9a08ed18b85 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:51.903775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.904613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.905048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.905406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.906422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=52a0bdcaee774dfba031cea6614afff3) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:51.906814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.909043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.915967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 63/133] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/role_assignments?user.id=52a0bdcaee774dfba031cea6614afff3&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:51.955385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:51.956050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:51.956343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:51.956629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:51.957390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:51.957707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:51.959220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:51.974081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 71/134] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:51.980055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:52.098287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:52.109630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c16f06-9b32-47d5-94f2-569f345edaa7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:52.160338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:52.161002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:52.161293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:52.161580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:52.163607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:52.165152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:52.166925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 64/135] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:51 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:52.173389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:52.300237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2738b9-de38-475c-910f-b5172e01f155 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:52.326964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:52.345926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:52.346717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:52.347011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:52.347300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] PATH_INFO: `/v3/projects/ce7940ca188e4a71a0b113e81dfddf85` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:52.348580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] RBAC: Authorizing `identity:get_project(project_id=ce7940ca188e4a71a0b113e81dfddf85)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:52.350019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88aef26-ce93-4c49-b7ab-4001cb18f766 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:52.351751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 72/136] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:52 2022] GET /identity/v3/projects/ce7940ca188e4a71a0b113e81dfddf85 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:52.358172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:52.500559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca57ab-c284-416a-a943-0a85fb0073d4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:52.547986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:52.548679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:52.548969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:52.549254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] PATH_INFO: `/v3/projects/ce7940ca188e4a71a0b113e81dfddf85/users/59c9455e874f42308d26ae107456be33/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:52.551287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] RBAC: Authorizing `identity:create_grant(project_id=ce7940ca188e4a71a0b113e81dfddf85, user_id=59c9455e874f42308d26ae107456be33, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:52.552742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b3de7-e0f5-4d3e-b345-9c678a4f829b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:52.640256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 65/137] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:40:52 2022] PUT /identity/v3/projects/ce7940ca188e4a71a0b113e81dfddf85/users/59c9455e874f42308d26ae107456be33/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:52.811647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37670180-37ef-4391-a42c-c9e593578565 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:52.812194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37670180-37ef-4391-a42c-c9e593578565 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:52.812194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37670180-37ef-4391-a42c-c9e593578565 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:52.813633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 73/138] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:52.819460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:52.819924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:52.820021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:52.846382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:52.846382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:52.873273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:52.941498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 66/139] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:52.979757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37670180-37ef-4391-a42c-c9e593578565 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:53.104181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37670180-37ef-4391-a42c-c9e593578565 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:53.147287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:53.148010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.148303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.148637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.163802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:53.169565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:53.170439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:53.171430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 74/140] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:53.178514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:53.316074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2463b1-df01-4ce0-928f-f2e866fec86d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:53.358394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:53.359048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.359337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.359621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.360361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:53.360671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:53.362076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:53.371903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 67/141] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:53.377706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:53.462477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:53.527168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c90899-74ba-4ac3-96fc-7eb50e58a157 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:53.571457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63888205-fea3-429a-9030-7222676313be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:53.572035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888205-fea3-429a-9030-7222676313be None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.572321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888205-fea3-429a-9030-7222676313be None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.572665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888205-fea3-429a-9030-7222676313be None admin] PATH_INFO: `/v3/users/5580a3ecfd9d47fb8a889209b62e0089` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.573917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63888205-fea3-429a-9030-7222676313be None admin] RBAC: Authorizing `identity:get_user(user_id=5580a3ecfd9d47fb8a889209b62e0089)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:53.575478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63888205-fea3-429a-9030-7222676313be None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:53.577296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 75/142] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:53 2022] GET /identity/v3/users/5580a3ecfd9d47fb8a889209b62e0089 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:53.582729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:53.603101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:53.700342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe458-d6e2-48b6-830b-bc6bf594001a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.700746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe458-d6e2-48b6-830b-bc6bf594001a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.701151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe458-d6e2-48b6-830b-bc6bf594001a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.702854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 76/143] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:53.708874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.709166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.709454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.733115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7146b0-de58-400d-92a2-d5abbf7d7b43 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:53.737239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:53.741122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:53.815176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:53.815843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:53.816140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:53.816425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:53.902406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 77/144] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:40:53.946675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:53.998782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:54.000311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:54.037656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:54.067542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:54.216853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070513a-94d7-4db2-bcbf-63115f4c1899 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:54.263714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 68/145] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:54.269610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:54.305787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:54.306574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:54.306865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:54.307150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:54.308645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:54.310300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:54.312610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 78/146] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:54.318263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:54.478756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25952a95-d6b9-4312-b62f-c35f0943f5fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:54.506990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385fb2-31bc-4a46-b46c-a1d0cf4e542b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:54.509533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:54.539367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:54.547389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:54.548074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:54.548372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:54.548665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:54.549957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:54.551589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:54.555524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 69/147] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:54 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:54.561971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:54.603139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:54.603795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:54.604085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:54.604373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:54.605100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:54.607237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1550e0-4b44-4732-be6e-ca6a9b43ef8b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:54.717773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ca3c-331d-4b38-86b5-46f1f43823ce None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:54.736142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 79/148] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:40:54 2022] POST /identity/v3/users => generated 283 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:40:54.768589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:54.769246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:54.769550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:54.769840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:54.770716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=5580a3ecfd9d47fb8a889209b62e0089) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:54.771040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:54.772587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e68139-ae6a-4ce4-b486-174ee891384f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:54.783769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 70/149] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:54 2022] GET /identity/v3/role_assignments?user.id=5580a3ecfd9d47fb8a889209b62e0089&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:55.008225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657501-4dfe-4cad-addb-9a8b738b2d87 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.008704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657501-4dfe-4cad-addb-9a8b738b2d87 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.008704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657501-4dfe-4cad-addb-9a8b738b2d87 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.010208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 80/150] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:55.016302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.017216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.017703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.039502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:55.041087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:55.072539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f1bbe-9d8b-49eb-8a6c-0a2996e9dcb1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.073171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f1bbe-9d8b-49eb-8a6c-0a2996e9dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.073651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f1bbe-9d8b-49eb-8a6c-0a2996e9dcb1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.075897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 81/151] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:40:55.084670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.085285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.085888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.108573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:55.114144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:55.115757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:55.153570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 71/152] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:55.191065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.221675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 82/153] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:55.248231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.334086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbef51b-d2fa-46f2-bccd-8ee8ac4dd840 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:55.391404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f499f9-f7c9-406f-b51d-ace022fba9f5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:55.393998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:55.395754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:55.396612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.396612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.396877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.419532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:55.424963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:55.425812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:55.426606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 72/154] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:40:55.439092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.450478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:55.451279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.451574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.455059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.466203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:55.471317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:55.472393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:55.473197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 83/155] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:40:55.479118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.499002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:55.578030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e586bb-5355-4c42-9bf4-e913ae5f8315 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:55.614599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1bdd1-5687-4f43-b58c-c83cd304da64 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:55.625657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:55.626713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.627009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.627296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.628375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:55.628983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:55.630602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a2064-669b-4f54-b56d-33734d201027 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:55.641824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 73/156] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:55.642997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151a81-4cdd-4691-9d55-51fff25f52bb None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.643293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151a81-4cdd-4691-9d55-51fff25f52bb None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.643577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151a81-4cdd-4691-9d55-51fff25f52bb None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.644823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 74/157] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:55.649802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707109-7221-43a3-b33a-040ae1f15793 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.650103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707109-7221-43a3-b33a-040ae1f15793 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.650628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca707109-7221-43a3-b33a-040ae1f15793 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.651790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 75/158] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:55.653398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca707109-7221-43a3-b33a-040ae1f15793 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.662744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:55.663422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.663713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.664528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.665288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:55.665605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:55.667092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf80af-ecbf-43bf-a191-f85c29ff2945 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:55.681727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 84/159] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:40:55.682578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b1b4-c605-46eb-826e-9546a003cc88 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.682578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b1b4-c605-46eb-826e-9546a003cc88 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.682757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b1b4-c605-46eb-826e-9546a003cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.703515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6317b1b4-c605-46eb-826e-9546a003cc88 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:55.704931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6317b1b4-c605-46eb-826e-9546a003cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:55.725801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:55.795289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 85/160] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:55.796655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024f41-8704-464d-849f-f9966543aa64 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.797035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024f41-8704-464d-849f-f9966543aa64 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.797220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024f41-8704-464d-849f-f9966543aa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.803873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca707109-7221-43a3-b33a-040ae1f15793 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:55.806717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:55.818082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-15024f41-8704-464d-849f-f9966543aa64 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:55.819454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-15024f41-8704-464d-849f-f9966543aa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:55.866334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:55.866966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:55.868592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:55.868592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:55.869486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:55.871004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:55.873052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 76/161] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:55.873689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:55.922238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 86/162] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:55.922950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024f41-8704-464d-849f-f9966543aa64 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.000583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6a81-effe-4c31-b34c-e7a0942297a1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.065163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.070326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.071069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.071357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.071687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] PATH_INFO: `/v3/users/52a0bdcaee774dfba031cea6614afff3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.073532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] RBAC: Authorizing `identity:get_user(user_id=52a0bdcaee774dfba031cea6614afff3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:56.075201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:56.079179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 77/163] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/users/52a0bdcaee774dfba031cea6614afff3 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:56.080004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.099681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024f41-8704-464d-849f-f9966543aa64 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.139014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.198369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.199057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.199484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.199844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.208247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e846bf-00d0-4fc8-94b8-c8ee80fb8c8e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.231841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:56.236954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:56.237814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 87/164] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:40:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:56.239293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.250967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.251917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.252398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.252868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.332695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:56.334744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:56.374949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.447615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 78/165] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:56.449091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.495440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f79e-2e9c-4bab-b85f-352fadd3d6dc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.532410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.533066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.533355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.533672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.550649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:56.555697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:56.556561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:40:56.557332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:56.558632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 88/166] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:56.563515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.594750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.644217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9f4c-12b3-4043-85ce-115aa10b0ded None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.694122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c2bbc-69f3-40f1-94b3-3f5be4042eb9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.725780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.726343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.726726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.727047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.728533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:56.732271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:56.739393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 79/167] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:56.740355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.740928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.741032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.741282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.741566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.742308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:56.742697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:56.744129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:56.754011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 89/168] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:56.754893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:56.762890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.830767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:56.870540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62104-642b-47ef-8563-817f53a657f2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:56.907380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:56.908254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:56.908954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:56.909976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:56.910882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:56.911342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:56.912910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:56.922881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 90/169] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:56.923880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.006947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589b442-0529-4f43-84a7-ff438bbf7ae7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.029323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63571104-1085-4ecf-9340-b61d0e919da6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.066132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:57.068747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.069662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.070115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.070596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/users/52a0bdcaee774dfba031cea6614afff3/roles/1c275785f4c947c994d1a4ef788661d3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.073561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, user_id=52a0bdcaee774dfba031cea6614afff3, role_id=1c275785f4c947c994d1a4ef788661d3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.075688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.098731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.100076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.100515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.101000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.102714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.104317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.107341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 80/170] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:56 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:57.108338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.173961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 91/171] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:40:56 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/users/52a0bdcaee774dfba031cea6614afff3/roles/1c275785f4c947c994d1a4ef788661d3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:57.175116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.206854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:57.247372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0f664-6ffc-4a5b-a299-276c78e09514 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.289563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.290985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.296587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.297278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.298887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.300699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.302981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 81/172] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:57 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:57.304074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.459839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13374b9e-0394-4397-b74d-aa452e524075 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.501679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.502764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.503269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.503764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.504787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:57.505310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.507343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.514738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 82/173] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:57 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:57.516528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.534752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:57.620545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bfa32-4ffa-450e-9173-5ca9ece8844b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.689556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.690283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.690617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.690905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] PATH_INFO: `/v3/users/59c9455e874f42308d26ae107456be33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.692267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] RBAC: Authorizing `identity:get_user(user_id=59c9455e874f42308d26ae107456be33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.693809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.695550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 92/174] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:57 2022] GET /identity/v3/users/59c9455e874f42308d26ae107456be33 => generated 275 bytes in 521 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:57.702860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:57.720217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bf9e-36ee-47bd-a863-d71bee028b44 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.785191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.785940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.786387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.786612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:57.790740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:57.842423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c94d-df99-41c4-ad1d-26c64d8ffdf3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:57.892765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:57.894304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede6bc2-73fe-435d-ba7b-6c80406e8e74 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:57.897455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:57.897922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:57.899345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:57.899711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:57.900093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.004954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 83/175] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:58.006262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b0bb-472c-4822-a7ac-b81276d6b063 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.006606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b0bb-472c-4822-a7ac-b81276d6b063 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.006894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b0bb-472c-4822-a7ac-b81276d6b063 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.008183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 84/176] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:58.010803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8b0bb-472c-4822-a7ac-b81276d6b063 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.016821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.018332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0b132-99bb-4d18-9c2a-3a95edf0934a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:58.126370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.166856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8b0bb-472c-4822-a7ac-b81276d6b063 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.178444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.213097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 93/177] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:58.218374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.218374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.218374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.222283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.223343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.223928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.224208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.229598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.231780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:58.233551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 85/178] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:58.234342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.236601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:58.238657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:58.348757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 94/179] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:58.349710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.384433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7164a7-cfd5-4ddd-8efa-a665f83000e6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.421641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.422216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.422539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.422822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] PATH_INFO: `/v3/projects/ce7940ca188e4a71a0b113e81dfddf85` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.424014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] RBAC: Authorizing `identity:get_project(project_id=ce7940ca188e4a71a0b113e81dfddf85)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.425468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cb744-cf7e-4391-bca2-d8b721d09050 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:58.427227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 86/180] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/projects/ce7940ca188e4a71a0b113e81dfddf85 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:58.428377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164a8-a7e4-4843-95b2-0926629e4476 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.428664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164a8-a7e4-4843-95b2-0926629e4476 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.428945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a164a8-a7e4-4843-95b2-0926629e4476 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.430319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 87/181] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:58.435801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ef407-50f1-4250-a4e3-8e59c524cc72 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.436090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ef407-50f1-4250-a4e3-8e59c524cc72 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.436390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ef407-50f1-4250-a4e3-8e59c524cc72 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.438387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 88/182] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:40:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:58.438387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ef407-50f1-4250-a4e3-8e59c524cc72 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.448523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.463818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7eea8f-9eac-4132-a403-d56d9b27a10c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.483875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.507243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.508455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.509134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.509564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.510548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:58.511032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.513410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:58.521570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 95/183] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:58.522997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.552425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ef407-50f1-4250-a4e3-8e59c524cc72 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.610197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.611013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.611374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.612039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.630209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-448c842f-7740-4222-b8aa-5331f878e2d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:58.636849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:58.638436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:40:58.639418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:40:58.642474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 89/184] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:58.644000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.644508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.644964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.674901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:58.676577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:58.677746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9e17-e9f9-4e14-a379-6d2028ea779b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.726434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.727168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.727455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.727741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.728524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=ce7940ca188e4a71a0b113e81dfddf85, user.id=59c9455e874f42308d26ae107456be33) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:58.728835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.730281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93273f6-5f9b-4046-bcc6-800ab606f6ac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:58.742395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 96/185] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/role_assignments?user.id=59c9455e874f42308d26ae107456be33&scope.project.id=ce7940ca188e4a71a0b113e81dfddf85&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:58.742395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.742395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.742395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.767026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 90/186] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:58.767329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:58.767739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.768653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:58.791768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.828826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:58.891128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 97/187] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:40:58.892246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:58.940870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf751447-c1c7-47d3-84b8-9e9e63440ec4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:58.989983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:58.990897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:58.991330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:58.991881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:58.992763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:58.993300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:58.994972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.005050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 91/188] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.009842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.025629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a800831-50a3-45bd-aed6-2cb1ff025219 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.070799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.071756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.072222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.072678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.086597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:40:59.093421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:40:59.094454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:40:59.095926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 98/189] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:40:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:40:59.096854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.106446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.125353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.133814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e80f-e369-4a94-81aa-a44fe6cd429b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.171688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.172318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.172602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.172957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.174195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.175606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.177382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 92/190] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.179537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.212889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df132668-8bb1-4a8e-beec-1ff6fd32f947 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.251302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.251947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.252234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.252516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.266985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.268513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.270388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 99/191] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.275569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.290705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4684da-8959-4fe9-91d7-fe3816ddd1f6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.329938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.330844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.331281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.331549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.332326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:59.332706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.334058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.341951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 93/192] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.342255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.343151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.374058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.388276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d4563-a201-434e-8c71-c528a1894fe8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.425226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.425883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.426172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.426504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.427237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.428698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.459855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ffd17-34ed-42ba-a99a-2d9b83153833 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.496869 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.497543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.497835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.498121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.542698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 100/193] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:40:59 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:59.544156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.582992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.585042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.614587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.675913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496a869-8245-41de-a471-c236e6e30c48 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.702088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.712969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 94/194] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:40:59.719520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.736004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.736725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.736963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.737247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.739273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.740894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.742640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 101/195] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.749343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.861452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9d40-e4e7-4881-bafb-7a8c7eeeb584 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.894496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:59.905984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc6c43-0876-4439-a04b-7454b893e88a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:59.918665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.919348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.919637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.919918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:59.921634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:59.923414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:59.925797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 95/196] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:59.931992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:59.956225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:59.956858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:59.957147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:59.957428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.026221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:00.045046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:00.046878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:00.064870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3007df2-7ce0-46e6-b379-307c9ad156d4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:00.107413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:00.108325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.108756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.109190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.110138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7e6b0fd3182649d4b82d2ed43ea7ca66) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:00.110622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:00.112233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d0dce-a958-4866-b235-2e3394339067 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:00.118756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 96/197] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/role_assignments?user.id=7e6b0fd3182649d4b82d2ed43ea7ca66&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:00.120242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3c7-7ef6-40af-88f0-ac55520f71e0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.120691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3c7-7ef6-40af-88f0-ac55520f71e0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.121644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed3c7-7ef6-40af-88f0-ac55520f71e0 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.123129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 97/198] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:00.135950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.136557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.137609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.175594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:00.177533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:00.179316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 102/199] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:41:00.186063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:00.298642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:00.319512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edbdc1-7e93-4833-a8a5-9d65706dc086 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:00.324458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:00.359966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 98/200] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:00.375175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:00.376133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.376560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.376991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.378330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:00.379972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:00.382105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 103/201] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:00 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:00.388037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:00.405300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:00.548244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d398fa-6cd7-48fd-98c8-4d291c050443 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:00.602600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:00.604738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.607174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.607670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.614122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed089f-fdc4-4212-b279-34a4a3d7bf40 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:00.632120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:00.643058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:00.643950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:00.644733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:00.650534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 104/202] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:00.654402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:00.688740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:00.696938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:00.697860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.699282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.701297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.702305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7e6b0fd3182649d4b82d2ed43ea7ca66) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:00.705371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:00.705371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:00.711692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 99/203] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:00 2022] GET /identity/v3/role_assignments?user.id=7e6b0fd3182649d4b82d2ed43ea7ca66&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:00.842963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065843bc-84ce-4861-981d-48a055458fdb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:00.898649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:00.899274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:00.899608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:00.899932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:00.900739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:00.901055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:00.903216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:00.918788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 105/204] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:00.924900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:01.022770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:01.096526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dac706-9ea4-4c19-acbd-e175e7b876f9 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:01.138605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:01.139273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:01.139556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:01.139831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] PATH_INFO: `/v3/users/52a0bdcaee774dfba031cea6614afff3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:01.141044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] RBAC: Authorizing `identity:get_user(user_id=52a0bdcaee774dfba031cea6614afff3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:01.142523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:01.144419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 100/205] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:00 2022] GET /identity/v3/users/52a0bdcaee774dfba031cea6614afff3 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:01.150138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:01.314991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:01.345817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b73a0-7775-462f-8d05-f55336bfeb07 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:01.397495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:01.398141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:01.398453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:01.398733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:01.489673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:01.491387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:01.579322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:01.606402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 106/206] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:01.611931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:01.759430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9f2ec-c182-42f5-8dcc-5e121b7f5cfd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:01.780580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:01.798526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:01.799198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:01.799486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:01.799769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:01.800962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:01.802463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aade96-d40c-4623-add0-8724b007bc1f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:01.804224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 101/207] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:01 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:01.811338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:01.976542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750c1b-2766-4fa6-bd0a-ed01bb64bca6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:02.017203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:02.018216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.018733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.019240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.020279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, scope.project.id=28ac5424310548079b28ca6f050683cf, user.id=52a0bdcaee774dfba031cea6614afff3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:02.020823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:02.022537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519204-c9f4-496e-96fc-2885f9161b86 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:02.037664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 107/208] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:01 2022] GET /identity/v3/role_assignments?user.id=52a0bdcaee774dfba031cea6614afff3&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:02.088991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d067-d3b4-4337-b01b-543e75c554a0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.089279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d067-d3b4-4337-b01b-543e75c554a0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.089664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d067-d3b4-4337-b01b-543e75c554a0 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.091670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 102/209] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:02.108448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.109035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.109500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.141142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:02.142692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:02.201660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:02.246595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 108/210] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:02.267353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d067-d3b4-4337-b01b-543e75c554a0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:02.390233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d067-d3b4-4337-b01b-543e75c554a0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:02.428670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:02.429330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.429667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.429957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.443625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:02.449312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:02.450317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:02.451132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:02.453370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 103/211] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:02.458970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:02.577231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017b14e-0ed7-42b8-8651-9584f49ce5bf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:02.621767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:02.622462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.622658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.622935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.623670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:02.624041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:02.625450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518aec1-e615-45c2-ab1f-799f7979a44e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:02.634596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 109/212] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:02.640710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:02.662625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec59-035d-4833-b600-ea6f61a04e36 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.663300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec59-035d-4833-b600-ea6f61a04e36 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.663891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec59-035d-4833-b600-ea6f61a04e36 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.665576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 110/213] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:02.672128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d764ea-e82b-4cd0-97ce-bf3f574e03a7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.672693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d764ea-e82b-4cd0-97ce-bf3f574e03a7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.673194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d764ea-e82b-4cd0-97ce-bf3f574e03a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.692375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-20d764ea-e82b-4cd0-97ce-bf3f574e03a7 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:02.693940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-20d764ea-e82b-4cd0-97ce-bf3f574e03a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:02.699808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:02.734726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:02.770259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 111/214] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:02.777926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe573a-f713-46ec-85b6-26ecca05d6fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:02.778005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291716-9ffe-409d-a031-af56aa7835c3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.778005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291716-9ffe-409d-a031-af56aa7835c3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.778005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291716-9ffe-409d-a031-af56aa7835c3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.779625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 112/215] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:02.785754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.786044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.786828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.811634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:02.812833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:02.813129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:02.813780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:02.814064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:02.814396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] PATH_INFO: `/v3/users/ddaa672fa1d541efb1eb57339c2d646e` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:02.830001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] RBAC: Authorizing `identity:get_user(user_id=ddaa672fa1d541efb1eb57339c2d646e)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:02.831471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:02.839332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 104/216] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:02 2022] GET /identity/v3/users/ddaa672fa1d541efb1eb57339c2d646e => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:02.839332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:02.902029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 113/217] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:02.904199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:02.961924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6daa-992f-48d4-b20f-5a84c59e0ade None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:02.963860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:02.998978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:02.999770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.000599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.000876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.012220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:03.017396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:03.018302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:03.019216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:03.019954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.020037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 105/218] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:03.020037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.039550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb414a9c-fd9c-41bf-b901-b6e341c0e5cf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.078045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.078695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.078981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.079258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.127311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd068eab-67b1-412c-b06c-556fb3d8db93 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.154410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.155888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d148a-45ec-45cc-8ce6-c2a318c7bde4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.163695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.164370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.164660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.164941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.166097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.168196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.169917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 106/219] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:03.174891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.202218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.251244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.260570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 114/220] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:03.261950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adf65-0ece-4e4a-b327-29313e8d6ddd None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.262237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adf65-0ece-4e4a-b327-29313e8d6ddd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.262550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adf65-0ece-4e4a-b327-29313e8d6ddd None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.263868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 115/221] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:03.264555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adf65-0ece-4e4a-b327-29313e8d6ddd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.276195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea52b0-cef1-47e6-a24a-6ed24a4d1ffd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.313962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.314608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.314888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.315160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.315878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:03.316208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.317588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.326107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 107/222] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:03.327096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.377859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adf65-0ece-4e4a-b327-29313e8d6ddd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.411899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.412529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.412812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.413088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.413783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.415265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.422949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.431517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c9ea5-f6fe-4d7b-8577-a7dd5a42c34c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.466296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.467056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.467284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.467565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.468739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.470194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab51850-00ee-491f-99d7-04d04f928b71 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.471915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 108/223] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:03.473055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.473334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.473615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.493455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:03.497061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:03.524714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 116/224] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:03 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:03.525556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.578714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 109/225] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:03.579625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.602279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.664949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c5715-8865-4828-885c-c2edce04774a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.698070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.708830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.709362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65869993-fe10-4e90-940b-d382a99e8506 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.709533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.710149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.710149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.711305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.713277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.716189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 117/226] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:03.717260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.757285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.757922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.758199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.758832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.759331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=ddaa672fa1d541efb1eb57339c2d646e) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:03.759730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.761079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.768174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 110/227] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/role_assignments?user.id=ddaa672fa1d541efb1eb57339c2d646e&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:03.769035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.876608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563166d-3605-4cb8-84e4-2e784ae4e0d0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.899363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:03.934886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95140c-af91-449e-90dc-dc1d3cc385eb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:03.940925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.942021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.942550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.943032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.960579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:03.968129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:03.969284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:03.970271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:03.972713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 118/228] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:03.979180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:03.990122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:03.991108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:03.991860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:03.992318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:03.995647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:03.998272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:03.998272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 111/229] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:04.004353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:04.042867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:04.176337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a20a-9b3f-475f-afcb-ccb6e5e709ce None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:04.200770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89b101-75f5-4e1b-907d-336628f6fb3b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:04.257155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:04.257800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:04.258090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:04.258392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:04.259139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:04.259452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:04.260869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:04.278832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 119/230] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:04.284855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:04.291862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:04.293150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:04.293666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:04.293958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7e6b0fd3182649d4b82d2ed43ea7ca66/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:04.296910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user_id=7e6b0fd3182649d4b82d2ed43ea7ca66, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:04.298434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:04.363777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:04.393146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 112/231] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:04 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7e6b0fd3182649d4b82d2ed43ea7ca66/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:41:04.442475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066a78a-7728-4f19-bc59-abc87462573f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:04.487096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:04.488096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:04.488390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:04.488698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:04.489944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:04.491740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:04.493584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 120/232] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:04 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:04.500560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:04.652993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:04.664865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39584c-77f8-4eca-9a77-dfa2ca763bf6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:04.703665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:04.704415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:04.704571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:04.704856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:04.706050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:04.707862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46ba65-988a-4c6a-8310-e3c031dc8c17 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:04.709855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 113/233] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:04 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:04.716371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:04.897456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eff80-755f-48b8-b974-25c82ea8377a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:04.905330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:04.940592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:04.941288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:04.941577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:04.941862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7e6b0fd3182649d4b82d2ed43ea7ca66/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:04.944183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user_id=7e6b0fd3182649d4b82d2ed43ea7ca66, role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:04.945654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9ddbc-8b4e-4f4f-b14c-6deba4ceade6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:05.051494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 121/234] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:04 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7e6b0fd3182649d4b82d2ed43ea7ca66/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:41:06.217694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6785543-d6ba-45b8-92db-ef89b1603e31 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.218189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6785543-d6ba-45b8-92db-ef89b1603e31 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.218189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6785543-d6ba-45b8-92db-ef89b1603e31 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.219791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 114/235] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:06.229065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.229568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.229819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.244498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff111c-a355-4c9b-9932-3bd31aca4179 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.244779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff111c-a355-4c9b-9932-3bd31aca4179 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.245767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff111c-a355-4c9b-9932-3bd31aca4179 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.248119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 115/236] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:06.253416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.253957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.254557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.255776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:06.257004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:06.286500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:06.287731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:06.400256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 116/237] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:06.403930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 122/238] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:06.427321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:06.438042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:06.545815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:06.571573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8f96f-e2d2-4408-8c49-fe16e7695944 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:06.604990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:06.617103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcd8fe-7e62-4475-84c6-2668f329ef95 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:06.617985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:06.618744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.619031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.619310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.630847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-2c29bb24-0a1a-4a70-b4de-e780f649c35c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:06.635840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:06.637082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:06.638364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 117/239] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:06.639042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e44f7-0f5a-4618-83ce-db396b9df005 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.639042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e44f7-0f5a-4618-83ce-db396b9df005 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.639450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e44f7-0f5a-4618-83ce-db396b9df005 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.644044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 118/240] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:06.648259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e44f7-0f5a-4618-83ce-db396b9df005 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:06.665375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:06.666407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.666517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.666935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.668531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:06.670136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f5ddb-7d5a-4ab0-9745-5749d758d608 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:06.672537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 123/241] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:06.673834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.674123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.674417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.697146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:06.698458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:06.786074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 124/242] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:06.787384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:06.812226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e44f7-0f5a-4618-83ce-db396b9df005 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:06.867019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:06.867943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.868077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.868394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.869133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:06.869444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:06.870898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:06.887100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 119/243] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:06.887100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:06.887100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:06.924787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:06.940433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07638bca-8e37-4600-b583-2c8f48f6155c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:06.983537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:06.984173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:06.984460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:06.984740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:06.985430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:06.986950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.007516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c11093-1682-43b9-b931-3fd4e3b0f6c6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.044377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.045078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.045378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.045930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.056784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-b3265588-3a4d-4cb8-849a-05b707fc9df8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:07.063279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:07.064592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 120/244] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:07.065419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d688-7313-479b-b133-425ecd2b52a7 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.066208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d688-7313-479b-b133-425ecd2b52a7 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.066208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05d688-7313-479b-b133-425ecd2b52a7 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.070039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 121/245] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:07.070837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05d688-7313-479b-b133-425ecd2b52a7 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.106729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 125/246] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:06 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:07.106729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.156475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.213971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05d688-7313-479b-b133-425ecd2b52a7 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.243762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b0971-4a13-4d8e-bfdc-83d49c1b7424 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.268111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.269732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.270340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.270995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] PATH_INFO: `/v3/users/7b098f9ab30f458f8a42fb43a1c71532` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.291708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.292571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.294088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.294088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.294256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] RBAC: Authorizing `identity:get_user(user_id=7b098f9ab30f458f8a42fb43a1c71532)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:07.294427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:07.294427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:07.295794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf518e-fe56-40d5-aa82-c9b8cd808740 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.295985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.298016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 122/247] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/users/7b098f9ab30f458f8a42fb43a1c71532 => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:07.299549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.299838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.300120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.306113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 126/248] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:07.307011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.323403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:07.326830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:07.328437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.443984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1e97-228f-43f3-af52-cf8728aff05a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.450030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 123/249] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:07.451208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.494321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.495250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.495562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.495897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.522335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.583367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:07.584776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0162cc-979a-4f42-b7f1-5b562d7e61aa None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.585949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e95bee-d649-4712-8773-510dcc9b782b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.626131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.641329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.642141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.642568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.642965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] PATH_INFO: `/v3/users/ddaa672fa1d541efb1eb57339c2d646e` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.644351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] RBAC: Authorizing `identity:get_user(user_id=ddaa672fa1d541efb1eb57339c2d646e)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:07.646010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.649010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 124/250] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/users/ddaa672fa1d541efb1eb57339c2d646e => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:07.649938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.699253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 127/251] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:07.700727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d19e17-bfad-4f98-99a6-3cba11066d46 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.701055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d19e17-bfad-4f98-99a6-3cba11066d46 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.701416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d19e17-bfad-4f98-99a6-3cba11066d46 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.703298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 128/252] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:07.704144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d19e17-bfad-4f98-99a6-3cba11066d46 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.789064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d06d-4f2c-47af-ab6c-a77e3cb6a498 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.815740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.829367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d19e17-bfad-4f98-99a6-3cba11066d46 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:07.835577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.836256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.836618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.836909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.848421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:07.850963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:07.856186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:07.857283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:07.858069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:07.861829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 125/253] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:07.861829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.871103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:07.872086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.872431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.872780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.874049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:07.875684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6368286-4399-49bd-be9f-2826a6623889 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:07.877511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 129/254] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:07.878841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:07.879196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:07.879636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:07.899338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:07.900824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:07.974916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 130/255] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:07.978863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:07.986373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca45c1-20e4-424b-aa13-cabd3472c8a4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.028382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.029042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.029330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.029612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.030881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.032382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.034423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 126/256] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:08.035220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.086885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.102647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16795cb-05db-4cdc-ac19-985fac38c4d4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.115548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.141020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.141736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.142022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.142125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb173b9-ef7d-4fcd-af22-ac5c74eaf6a2 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.142298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.143111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:08.143420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.144923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.155857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 131/257] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:08.156520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.180575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.181291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.184396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.184396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/ddaa672fa1d541efb1eb57339c2d646e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.184553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user_id=ddaa672fa1d541efb1eb57339c2d646e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.186056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.273019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 127/258] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:08 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/ddaa672fa1d541efb1eb57339c2d646e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:08.274054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.282637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7552596-3d09-4517-97a1-ed05e356723d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.331080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.332311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.332692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.332979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.348955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:08.356614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:08.359175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 132/259] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:08.361114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.399966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.488703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dc43-b530-4591-96d5-18095c891ebb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.535190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.535860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.536290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.536572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.538033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.540345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.543310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 133/260] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:08.544429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.546249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1bfbd-4c58-4479-95a2-82c475bdad75 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.559824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.595171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.595816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.596103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.596389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.597205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7b098f9ab30f458f8a42fb43a1c71532) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:08.597527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.599086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.606617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 128/261] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/role_assignments?user.id=7b098f9ab30f458f8a42fb43a1c71532&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:08.607648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.686510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.690545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317f61-4225-4ae9-9c7e-750fc76bed5b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.741007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.741655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.741941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.742220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.743193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:08.743555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.745439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.757547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 134/262] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:08.764225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:08.817374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20177c-979a-4d89-860a-cd357f73cc3c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.861662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.862218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.862529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.862816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.903077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb81ab1-718f-4ead-9d47-15f009d52fc2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:08.908095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:08.945204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.946957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.949747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:08.950471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:08.950834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:08.951121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] PATH_INFO: `/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:08.952336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6b0fd3182649d4b82d2ed43ea7ca66)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:08.953830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:08.955890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 135/263] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/users/7e6b0fd3182649d4b82d2ed43ea7ca66 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:08.976085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:09.030849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:09.060217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 129/264] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:09.066531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:09.184912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc062a-274f-4863-b0e2-8b69acfa1d41 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:09.186558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:09.229372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95953598-69da-47a3-b438-08e4a99aad18 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:09.267084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:09.267289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:09.267910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:09.268198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:09.268485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:09.268907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:09.269381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:09.269754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:09.270111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:09.271426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:09.273635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 130/265] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:09 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:09.299299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:09.350129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:09.352339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:09.385003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:09.460158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 136/266] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:09.466778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:09.524427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda425e3-805d-431f-a4ce-cca4f9cbea56 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:09.577017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45933de-bb94-4ef8-aa92-4598dd6239c4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:09.614650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:09.618254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:09.619326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:09.619804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:09.620272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:09.622656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:09.624343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:09.626312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 137/267] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:09 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:09.635236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:09.635689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:09.635893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:09.636802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:09.637122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:09.638378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7e6b0fd3182649d4b82d2ed43ea7ca66) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:09.638617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:09.640124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d6b26-1b3c-4518-9a70-456a74a64c12 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:09.661066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 131/268] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:09 2022] GET /identity/v3/role_assignments?user.id=7e6b0fd3182649d4b82d2ed43ea7ca66&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:09.791406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c340e-2670-45ed-af3c-19cf9ce382b8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:09.856988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:09.857636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:09.857985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:09.858270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:09.859179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7e6b0fd3182649d4b82d2ed43ea7ca66) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:09.859500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:09.861011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faaa09-1ea2-4935-922d-552182ebcaaa None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:09.873376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 138/269] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:09 2022] GET /identity/v3/role_assignments?user.id=7e6b0fd3182649d4b82d2ed43ea7ca66&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:10.100835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe093a-abe1-450b-a9db-ea05e4c78375 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.100835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe093a-abe1-450b-a9db-ea05e4c78375 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.100835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe093a-abe1-450b-a9db-ea05e4c78375 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.101558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 132/270] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:10.108109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.108402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.108746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.119782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625baf61-544e-4aad-b1a8-77a616758d61 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.120063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625baf61-544e-4aad-b1a8-77a616758d61 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.120349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625baf61-544e-4aad-b1a8-77a616758d61 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.122446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 133/271] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:10.127697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.127987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.128271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.134587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:10.135776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:10.138119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:10.153552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:10.154838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:10.224304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:10.237293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 139/272] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:10.254739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 134/273] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:10.271147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:10.277243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:10.413846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aac39-14d3-4d3a-95d7-1c42219c3301 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:10.420271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd227b-4ee0-49e4-ab78-5a84a48e63f5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:10.451317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:10.452043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.452330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.452612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.453834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:10.455352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:10.457256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 140/274] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:10.457725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:10.463622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:10.466807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:10.467587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.467944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.468302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.484933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:10.491208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:10.492104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:10.493037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 135/275] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:10.500050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:10.546696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:10.597406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014638-b3aa-4271-a3ee-e5db61fe5188 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:10.642973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13179d2a-e399-401c-8ecf-1402b199b83a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:10.644483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:10.645852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.646634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.647420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.649086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:10.652415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:10.681562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:10.682113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.682425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.682748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.684107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:10.684414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:10.685868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:10.698406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 136/276] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:10.709305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:10.750538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 141/277] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:10 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:10.787666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:10.842415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c66405-825e-40ea-bb2f-739049d573a9 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:10.885766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:10.886857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:10.887290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:10.887713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] PATH_INFO: `/v3/users/34ded6bec25d477b8f1c6cd612adfb4e` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:10.903516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=34ded6bec25d477b8f1c6cd612adfb4e)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:10.905287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8fc92-071b-49c6-847f-c59436f6a1c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:10.912628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 137/278] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:10 2022] GET /identity/v3/users/34ded6bec25d477b8f1c6cd612adfb4e => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:10.914311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:11.030471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cac5f7-aa0a-4c51-b9b8-8ccd0d62df17 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:11.054474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:11.069346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:11.070008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.070293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.070622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.071435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, user.id=34ded6bec25d477b8f1c6cd612adfb4e, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:11.071773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:11.073398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a8e74-0d0e-418d-a46d-54cd26a2db1e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:11.080610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 142/279] 10.222.0.85 () {56 vars in 1308 bytes} [Thu Jul 28 17:41:10 2022] GET /identity/v3/role_assignments?user.id=34ded6bec25d477b8f1c6cd612adfb4e&scope.system=all&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:11.223902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee804-cb13-4de8-8edc-df8cbc2a6e3a None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.224234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee804-cb13-4de8-8edc-df8cbc2a6e3a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.224474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ee804-cb13-4de8-8edc-df8cbc2a6e3a None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.226122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 138/280] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:11.235022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afca350-1a47-414f-8c7f-b442f6fcda53 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.235413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afca350-1a47-414f-8c7f-b442f6fcda53 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.235635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afca350-1a47-414f-8c7f-b442f6fcda53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.257996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8afca350-1a47-414f-8c7f-b442f6fcda53 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:11.259577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8afca350-1a47-414f-8c7f-b442f6fcda53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:11.335986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 143/281] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:11.361936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ee804-cb13-4de8-8edc-df8cbc2a6e3a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:11.472741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b983-c06b-49a8-a287-b809182db7d9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.473287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b983-c06b-49a8-a287-b809182db7d9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.473287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b983-c06b-49a8-a287-b809182db7d9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.474785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 144/282] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:11.480285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.480570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.480853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.500925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:11.502143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:11.545210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ee804-cb13-4de8-8edc-df8cbc2a6e3a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:11.577048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:11.630190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:11.630802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.631089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.631367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.653675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 145/283] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:11.655468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:11.660618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:11.661478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:11.662315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 139/284] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:11.669561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:11.688399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:11.740680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:11.805060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f545-497b-4c3f-8362-68da8556097e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:11.847781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:11.848872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.849454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.850183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.870952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b6ef-f63b-4256-b90b-644da071b481 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:11.872318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:11.878174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:11.879270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:11.880078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:11.892416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 140/285] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:11.897762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:11.927043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:11.927834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:11.928129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:11.928415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:11.929272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:11.929596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:11.931136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:11.938783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:11.953909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 146/286] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:11.959533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.012460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc123cf-edb8-4df9-8203-5d7f2cb7b6aa None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.050100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.051114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.051708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.052236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.053157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] RBAC: Adding query filter params (name=reader) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:12.053683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.055353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:12.064673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 141/287] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:12.071316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.127857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffea9-b672-4310-a0d1-b1026efc787c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.143438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:12.214593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eededce-015b-4b78-857e-d51f49a4ddae None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.218568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:12.239409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.240105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.240395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.240702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] PATH_INFO: `/v3/users/ddaa672fa1d541efb1eb57339c2d646e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.242015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] RBAC: Authorizing `identity:get_user(user_id=ddaa672fa1d541efb1eb57339c2d646e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.243568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:12.245412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 147/288] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:11 2022] GET /identity/v3/users/ddaa672fa1d541efb1eb57339c2d646e => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:12.252361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.301181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.302506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.303112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.303589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] PATH_INFO: `/v3/users/7b098f9ab30f458f8a42fb43a1c71532` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.305038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] RBAC: Authorizing `identity:get_user(user_id=7b098f9ab30f458f8a42fb43a1c71532)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.306712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:12.309970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 142/289] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:12 2022] GET /identity/v3/users/7b098f9ab30f458f8a42fb43a1c71532 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:12.319421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.426056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ee1af-9591-4a59-998c-89f34126475d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.464320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.465167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.465458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.465744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.582706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:12.592717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b49c9-33d0-4db9-be47-113c0d4b7823 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.602243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.605323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:12.691831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.692431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.692718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.692996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.694320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.695863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:12.697593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 143/290] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:12 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:12.704324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.734530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 148/291] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:41:12.741139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:12.752994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:12.883863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47f338-f68c-4ee4-939d-2a8bbad14730 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.885678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:12.951999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8aadb-5085-442c-bd0d-9c85722851d0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:12.956989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:12.957929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:12.962827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:12.963269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7b098f9ab30f458f8a42fb43a1c71532/roles/1c275785f4c947c994d1a4ef788661d3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:12.965648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user_id=7b098f9ab30f458f8a42fb43a1c71532, role_id=1c275785f4c947c994d1a4ef788661d3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:12.967310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4967a5-59a3-43ac-98d9-b1e21fb1f0d9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:13.002033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:13.003080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.003514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.003969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.005340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:13.006985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:13.009232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 149/292] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:12 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:13.015054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.046578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 144/293] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:12 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/users/7b098f9ab30f458f8a42fb43a1c71532/roles/1c275785f4c947c994d1a4ef788661d3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:41:13.150776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d992679-9468-4842-bc88-812b79c2df50 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:13.200159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:13.200806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.201098 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.201379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.202327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=ddaa672fa1d541efb1eb57339c2d646e) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:13.202681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:13.204329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d87e9-0d80-4ea5-98fc-7cd14c7054fc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:13.218310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 150/294] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:13 2022] GET /identity/v3/role_assignments?user.id=ddaa672fa1d541efb1eb57339c2d646e&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:13.376753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a3df-a301-46b0-8c04-bb73aa523789 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.377117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a3df-a301-46b0-8c04-bb73aa523789 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.377335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26a3df-a301-46b0-8c04-bb73aa523789 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.379039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 145/295] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:13.386175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61793b-67a8-4a1e-ae44-db843c3b2ded None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.386519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61793b-67a8-4a1e-ae44-db843c3b2ded None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.386783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61793b-67a8-4a1e-ae44-db843c3b2ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.393988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:13.410029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8d61793b-67a8-4a1e-ae44-db843c3b2ded None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:13.411306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8d61793b-67a8-4a1e-ae44-db843c3b2ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:13.500121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 151/296] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:13.527410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a3df-a301-46b0-8c04-bb73aa523789 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.581719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad31b8-4510-44e8-87c6-dfc75d29dc42 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.582015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad31b8-4510-44e8-87c6-dfc75d29dc42 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.582749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad31b8-4510-44e8-87c6-dfc75d29dc42 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.584389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 152/297] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:13.590125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.590748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.591219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.615026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:13.616598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:13.643787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:13.655037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26a3df-a301-46b0-8c04-bb73aa523789 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:13.699199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:13.699735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.700024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.700308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.704956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 153/298] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:13.717502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:13.722546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:13.723439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:13.724230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:13.725390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 146/299] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:13.731365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.736104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.813308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:13.865068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91890a-9ba8-493a-be5b-e9e2bb9a15ec None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:13.876490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41697f-130c-49f3-a5c9-50f9e0daa546 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:13.903054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:13.903513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.903801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.904081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.904924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:13.905237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:13.906803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb622e-28ed-4015-8744-ecad981d6e3f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:13.914962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:13.915611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.916017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.916348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.918220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:13.919807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:13.920876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 154/300] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:13.922839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 147/301] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:13.923265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c87dd-c4b0-4056-9d8e-8836bba561f2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:13.923265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c87dd-c4b0-4056-9d8e-8836bba561f2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:13.923265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c87dd-c4b0-4056-9d8e-8836bba561f2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:13.929707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 155/302] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:13.929849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.929954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c87dd-c4b0-4056-9d8e-8836bba561f2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:13.952421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.045128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d74597-2c51-4819-922f-67e9d6eaa0fa None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.056956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c87dd-c4b0-4056-9d8e-8836bba561f2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.083589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.084158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.084404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.084504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.084713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] PATH_INFO: `/v3/users/41672b50df4e4542a3d6ff5ee36ebd61` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.097112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.098160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.098160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.098424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] PATH_INFO: `/v3/groups` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.099078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.100524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:14.100623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] RBAC: Authorizing `identity:get_user(user_id=41672b50df4e4542a3d6ff5ee36ebd61)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.102183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb40b61-6a9c-435e-8ec4-9ebddec4217a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:14.109400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 148/303] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:13 2022] GET /identity/v3/users/41672b50df4e4542a3d6ff5ee36ebd61 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:14.109400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.109400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.109400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.124898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:14.126081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:14.194267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 149/304] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:14.198537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 156/305] 10.222.0.85 () {60 vars in 1134 bytes} [Thu Jul 28 17:41:13 2022] POST /identity/v3/groups => generated 229 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:14.198537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:14.222515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:14.235205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.353145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3bf-b70d-419d-96c0-a34df5205452 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.366712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769805d9-37de-4324-9478-d03150f6bb7e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.400370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.401103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.402000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.402317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.403166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, user.id=41672b50df4e4542a3d6ff5ee36ebd61, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:14.403484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.404929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:14.409652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.410413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 157/306] 10.222.0.85 () {56 vars in 1308 bytes} [Thu Jul 28 17:41:14 2022] GET /identity/v3/role_assignments?user.id=41672b50df4e4542a3d6ff5ee36ebd61&scope.system=all&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:14.410897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.411481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.411981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.426318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:14.431986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:14.432967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:14.433760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:14.436392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 150/307] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:14.442391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:14.509736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.573024 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180cc3f-dc65-4b85-846b-429e2d6725e3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.612455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.614011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.614555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.615043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.615983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:14.616513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.618241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:14.628374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 158/308] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:14.637145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:14.715039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.749430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c7d2f-0af5-49c1-8b1e-108089986aa6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.787194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.787861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.788159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.788430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] PATH_INFO: `/v3/users/34ded6bec25d477b8f1c6cd612adfb4e` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.789748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] RBAC: Authorizing `identity:get_user(user_id=34ded6bec25d477b8f1c6cd612adfb4e)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.791264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b299e9-c5e5-46a3-8b38-5040d58cca8c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:14.793780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 151/309] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:14 2022] GET /identity/v3/users/34ded6bec25d477b8f1c6cd612adfb4e => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:14.799811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:14.918070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794561e-5aa4-4f56-aa63-78b480a54a94 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:14.929208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:14.954417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:14.955102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:14.955393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:14.955683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] PATH_INFO: `/v3/system/users/34ded6bec25d477b8f1c6cd612adfb4e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:14.957311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=34ded6bec25d477b8f1c6cd612adfb4e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:14.958865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5683baa-8040-4044-999d-ecab86612af9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:15.029331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 159/310] 10.222.0.85 () {58 vars in 1334 bytes} [Thu Jul 28 17:41:14 2022] PUT /identity/v3/system/users/34ded6bec25d477b8f1c6cd612adfb4e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:15.466838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af26ac-0278-41b2-b474-c5266ad899c0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:15.467225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af26ac-0278-41b2-b474-c5266ad899c0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:15.467400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af26ac-0278-41b2-b474-c5266ad899c0 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:15.468978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 152/311] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:15.475266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:15.475917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:15.476431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:15.497334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:15.498904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:15.583820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 160/312] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:15.612233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af26ac-0278-41b2-b474-c5266ad899c0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:15.724455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af26ac-0278-41b2-b474-c5266ad899c0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:15.763930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:15.764654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:15.764935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:15.765210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:15.770680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:15.777325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:15.782216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:15.791136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:15.791922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 153/313] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:15.793129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:15.912468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d5999-c9ff-43de-9e67-9acbb5dca3ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:15.914872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:15.950227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:15.950897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:15.951181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:15.951454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:15.952174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:15.952475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:15.953865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:15.963003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 161/314] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:15.968993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.087940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0bd0-5e94-4f0f-9a0b-b573ca21445e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:16.127080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:16.127995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.128430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.128848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] PATH_INFO: `/v3/users/7b098f9ab30f458f8a42fb43a1c71532` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.130288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] RBAC: Authorizing `identity:get_user(user_id=7b098f9ab30f458f8a42fb43a1c71532)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:16.132044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f1b8a-baa5-4b62-b648-ddd8886b6367 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:16.134186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 154/315] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:15 2022] GET /identity/v3/users/7b098f9ab30f458f8a42fb43a1c71532 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:16.139941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.295592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac19da-4867-4f1c-af1c-b4d0a8c12828 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:16.334007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:16.334940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.335381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.335808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.374288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:16.415599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:16.418001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63c26-15ef-4fb9-885a-cf2a56c4cf55 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:16.473768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf73f-32fb-414c-a66f-64562d640e94 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.474757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf73f-32fb-414c-a66f-64562d640e94 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.475275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf73f-32fb-414c-a66f-64562d640e94 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.477296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 155/316] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:16.485840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455bd3f-4257-4814-8144-cc8a95c39a43 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.486124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455bd3f-4257-4814-8144-cc8a95c39a43 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.486422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455bd3f-4257-4814-8144-cc8a95c39a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.505242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d455bd3f-4257-4814-8144-cc8a95c39a43 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:16.506529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d455bd3f-4257-4814-8144-cc8a95c39a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:16.545536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 162/317] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:16.546857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2508-1311-4c84-a7d0-94e2efcf5dd7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.547149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2508-1311-4c84-a7d0-94e2efcf5dd7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.547428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2508-1311-4c84-a7d0-94e2efcf5dd7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.548846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 163/318] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:16.550722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f2508-1311-4c84-a7d0-94e2efcf5dd7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.576817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:16.593888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 156/319] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:16.595694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.595985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.596262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.633817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:16.635008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:16.650791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:16.691189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f2508-1311-4c84-a7d0-94e2efcf5dd7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:16.714603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 157/320] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:16.715746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.743633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:16.744393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.744769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.745706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.747083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:16.748561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:16.750291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 164/321] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:16 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:16.751329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.850690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2e0e7-80a2-4a54-8985-59c7d7c0905d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:16.880216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:16.893902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ceac9-f1a2-4c66-a5e6-e33996f4dfad None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:16.897348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:16.898012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.898297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.898610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.900446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:16.902858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:16.910083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 158/322] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:16.910083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:16.932199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:16.934104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:16.934418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:16.934703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:16.937026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:16.947778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:16.956908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:16.957831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 165/323] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:16.958708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.048937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f0d7-7b0c-4000-8414-ba82bbc89f2d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.081509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525623c-b0cc-464f-a13e-8b433e70a589 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.089414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.090049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.090331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.090638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.091442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, user.id=7b098f9ab30f458f8a42fb43a1c71532) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:17.091760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.093295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.099867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 159/324] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:16 2022] GET /identity/v3/role_assignments?user.id=7b098f9ab30f458f8a42fb43a1c71532&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:17.111446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.119993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.120831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.121239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.121723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] PATH_INFO: `/v3/groups` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.122538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.124194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb194aa-2d4d-434c-af7b-80f1f6506dd6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.186918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:17.235589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fc85-9114-43d0-bd35-ba0a880c1bd0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.263187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 166/325] 10.222.0.85 () {60 vars in 1134 bytes} [Thu Jul 28 17:41:16 2022] POST /identity/v3/groups => generated 226 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:17.263187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50dae-7cc1-4b9c-ae3c-c828884dedd1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.263187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50dae-7cc1-4b9c-ae3c-c828884dedd1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.263187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50dae-7cc1-4b9c-ae3c-c828884dedd1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.270256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 167/326] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:17.275859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.276249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.276499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.297953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.298497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.299093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.299093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.299591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:17.299850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:17.300159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.301381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:17.301669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.315658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 160/327] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:17.321646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.387349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:17.412898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 168/328] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:17.440266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.472655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809ff5c-3022-42aa-8e97-f515849fc8f4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.492097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:17.512592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.513245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.513543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.513828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] PATH_INFO: `/v3/users/41672b50df4e4542a3d6ff5ee36ebd61` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.515580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] RBAC: Authorizing `identity:get_user(user_id=41672b50df4e4542a3d6ff5ee36ebd61)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.517514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.519441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 161/329] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:17 2022] GET /identity/v3/users/41672b50df4e4542a3d6ff5ee36ebd61 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:17.526810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.575847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d31e2-4d98-42ef-b79c-887cb0f5f766 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.639012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.640096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.640581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.641055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.661140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:17.666682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:17.668931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:17.669738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 169/330] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:17.675660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.682547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40acf-1f4c-43e7-a22d-96a659b2ea56 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.736795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.737540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.737832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.738117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] PATH_INFO: `/v3/system/users/41672b50df4e4542a3d6ff5ee36ebd61/roles/1c275785f4c947c994d1a4ef788661d3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.740679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=41672b50df4e4542a3d6ff5ee36ebd61, role_id=1c275785f4c947c994d1a4ef788661d3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.742453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.744361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:17.817107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb920e0c-a207-4694-9df6-d84e9c8f3b1f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:17.827596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 162/331] 10.222.0.85 () {58 vars in 1334 bytes} [Thu Jul 28 17:41:17 2022] PUT /identity/v3/system/users/41672b50df4e4542a3d6ff5ee36ebd61/roles/1c275785f4c947c994d1a4ef788661d3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:17.865809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:17.866503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:17.866797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:17.867076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:17.867904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:17.868216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:17.869691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:17.882523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 170/332] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:17.888611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:17.963095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:18.038710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae375fe-b984-4227-a8b4-d8fa3622859b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:18.080665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:18.081326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:18.081613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:18.081890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] PATH_INFO: `/v3/users/34ded6bec25d477b8f1c6cd612adfb4e` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:18.083109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] RBAC: Authorizing `identity:get_user(user_id=34ded6bec25d477b8f1c6cd612adfb4e)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:18.084573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c8e7e-750e-4539-9004-5be0d9d036a9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:18.086547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 163/333] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:17 2022] GET /identity/v3/users/34ded6bec25d477b8f1c6cd612adfb4e => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:18.092892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:18.216937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcc9c6-dda2-4931-9e88-2bdad09e4f55 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:18.224938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:18.258582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:18.259269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:18.259563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:18.259911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:18.260778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] RBAC: Adding query filter params (role.id=ec41984b84ff479d8a0e7ad9405aa8b1, user.id=34ded6bec25d477b8f1c6cd612adfb4e, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:18.261178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:18.262694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7a4c3-8bbd-4110-897f-ed5733368682 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:18.279031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 171/334] 10.222.0.85 () {56 vars in 1308 bytes} [Thu Jul 28 17:41:18 2022] GET /identity/v3/role_assignments?user.id=34ded6bec25d477b8f1c6cd612adfb4e&scope.system=all&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:19.641227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44919-2f06-41e8-85a2-7365e4754d04 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:19.641748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44919-2f06-41e8-85a2-7365e4754d04 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:19.641872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c44919-2f06-41e8-85a2-7365e4754d04 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:19.643457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 164/335] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:19.649104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:19.649564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:19.649749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:19.678940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:19.680217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:19.769670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 172/336] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:19.803473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c44919-2f06-41e8-85a2-7365e4754d04 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:19.933686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c44919-2f06-41e8-85a2-7365e4754d04 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:19.954951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:19.977959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:19.978871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:19.979159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:19.979976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.000595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a2920a17-df96-4425-9f86-28b92f929427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:20.005711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:20.006824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:20.008376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 165/337] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:20.013816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.096219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903733-15dd-4452-8000-6ce788c08271 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.096560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903733-15dd-4452-8000-6ce788c08271 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.096776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903733-15dd-4452-8000-6ce788c08271 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.098145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 166/338] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:20.103541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.103854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.104144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.125614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:20.126843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:20.130718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:20.139196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510e29-6c6b-41fc-b5f8-a2b303556d08 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.185020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.185724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.186061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.186363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.191414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:20.191414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:20.191414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:20.202806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 173/339] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:20.208524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.210267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 167/340] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:20.236054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.349952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341eaac-9158-4d56-94fc-8d7aa39de9ad None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.358198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:20.365789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c26a6-470f-43ae-8c15-da7f1c4f119c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.390456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.391166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.391513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.391730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.404930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:20.409919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:20.412151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:20.414165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 168/341] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:20.421901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.424323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.424982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.425271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.425551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.454858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:20.505461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:20.507524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:20.536349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80675621-45a4-4605-81c5-0ec011ebab25 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.580338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.580980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.581262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.581542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.582280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:20.582650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:20.584068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:20.596312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 169/342] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:20.624520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.630764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 174/343] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:20.636858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.669551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:20.718625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:20.772166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a935c-6b84-416c-9ec9-cf89b22d293c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.773852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3a56-0678-4f5b-aa2c-9c6b9cba6cd7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:20.817880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.818569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.818856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.819134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.823763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:20.825501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:20.829220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 175/344] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:20.833723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:20.843250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:20.844204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:20.844504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:20.845109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:20.959968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:20.961467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:21.006411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:21.079787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b2922-4a24-43c7-98c0-6f854b8dadd3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.090801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 170/345] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:21.096394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:21.119207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:21.149599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:21.150389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.150671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.150950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.164794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:21.166281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:21.172360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 176/346] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:20 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:21.176382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:21.264647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200822-f9c7-4466-ac01-d77aa2671839 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.326482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8b5fe-46c9-4f6f-a3b2-911a84edeab9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.343317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:21.365831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:21.366571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.366857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.367137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.368736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:21.370901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda383d-9b25-4fda-ac1b-e9b07a5586e2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:21.371270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:21.372175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.372696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.373424 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 171/347] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:21 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:21.373743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.379157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=28ac5424310548079b28ca6f050683cf) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:21.379562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6247e-91d5-4661-bf9e-30dcdf24c5b1 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.380086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:21.380191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6247e-91d5-4661-bf9e-30dcdf24c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.380452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6247e-91d5-4661-bf9e-30dcdf24c5b1 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.382555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45eaa0-11d4-486f-bafc-9be8556a594d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:21.382770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 172/348] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:21.383549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6247e-91d5-4661-bf9e-30dcdf24c5b1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:21.390547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 177/349] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:21 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:21.392189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.392775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.393300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.437761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:21.439027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:21.547447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:21.608427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 178/350] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:21.640412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6247e-91d5-4661-bf9e-30dcdf24c5b1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.651653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:21.693953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:21.694928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.695254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.695540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.697898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:21.700669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd4143-a5c5-4114-84ab-8dc154401bb2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:21.702424 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 173/351] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:21 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:21.703636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1889-b080-4813-a731-ca796413eb96 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.703924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1889-b080-4813-a731-ca796413eb96 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.704209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1889-b080-4813-a731-ca796413eb96 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.707254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 174/352] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:21.712517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1889-b080-4813-a731-ca796413eb96 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:21.867447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:21.880797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df8dd0-837b-4753-9e0f-5fc67dc37d07 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.955334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1889-b080-4813-a731-ca796413eb96 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:21.959378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:21.960048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.960334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.960615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:21.981950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d542efdf-35ca-42b3-8c39-0fe21e44da4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:21.990561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:21.991494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:21.992296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:21.993373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 179/353] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:21.994529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5c6a-281c-45b6-9137-ac53b65af10c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:21.994857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5c6a-281c-45b6-9137-ac53b65af10c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:21.995143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5c6a-281c-45b6-9137-ac53b65af10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.021744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.022902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.023406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.023703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5f7f5c6a-281c-45b6-9137-ac53b65af10c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:22.024117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.025177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=28ac5424310548079b28ca6f050683cf) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:22.025693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:22.025795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5f7f5c6a-281c-45b6-9137-ac53b65af10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:22.028556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:22.040606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 175/354] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:21 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 260 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:22.041889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.119704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.177693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 180/355] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:22.179443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a096f2-1ce1-40b5-bd6d-c918dc55d289 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.179927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a096f2-1ce1-40b5-bd6d-c918dc55d289 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.181448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a096f2-1ce1-40b5-bd6d-c918dc55d289 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.187051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 181/356] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:22.189615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.190207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.190737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.211464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:22.213034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:22.237324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe14d5c-e439-4c71-8525-11346d14fcd7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.274645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.284363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:22.289556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:22.298432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 182/357] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:22.299203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 176/358] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:22.300018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.305150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.422523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb98d17-0dc2-4c2b-a3fb-828ed4baf010 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.428533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fad284-d6ec-4266-84bf-c53d1fd6bad8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.430388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.461899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.462666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.463019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.463376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.467103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.468092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.468600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.469070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.481299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:22.487605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:22.488613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:41:22.489376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:22.492091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 183/359] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:22.493091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.534934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.546536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:22.551955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:22.612139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a198f54-57bb-4a1c-95e3-923fa1dbbac8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.676663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.677255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.677542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.677822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.689855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 177/360] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:41:22.690761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.706789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:22.712410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:22.713320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 184/361] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:22.714663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.760350 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.831041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:22.846613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c6ab1-87a8-4341-b994-06d79f3c3e72 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.865634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851cfc-61f4-4ed2-9745-dcc69fa2d838 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:22.889474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.890504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.890930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.891349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.892225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:22.892678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:22.894249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:22.905700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 178/362] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:22.906935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:22.915974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:22.917207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.917508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.917786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.919620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:22.921819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8993d-3d68-4a31-aefd-acf6e11fbff4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:22.927936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 185/363] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:22.927936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87111b4-7e90-4ecd-b32e-bc0c9ee6ffac None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:22.927936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87111b4-7e90-4ecd-b32e-bc0c9ee6ffac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:22.927936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87111b4-7e90-4ecd-b32e-bc0c9ee6ffac None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:22.928256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 186/364] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:22.928788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87111b4-7e90-4ecd-b32e-bc0c9ee6ffac None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.050695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3486-240d-4412-a691-047d415cb573 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.056725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.070552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87111b4-7e90-4ecd-b32e-bc0c9ee6ffac None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.099627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.099925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.100334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.100745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.101856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:23.102232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.104751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8f2cd-4f9a-4875-a146-56dca8935112 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.112572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.113485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.113972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.114970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] PATH_INFO: `/v3/users/41672b50df4e4542a3d6ff5ee36ebd61` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.115570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 179/365] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:23.116624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] RBAC: Authorizing `identity:get_user(user_id=41672b50df4e4542a3d6ff5ee36ebd61)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.116884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.117178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.117688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.118941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.121758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 187/366] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:41:22 2022] GET /identity/v3/users/41672b50df4e4542a3d6ff5ee36ebd61 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:23.122795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.148793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:23.150733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:23.164357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.237632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 180/367] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:23.238651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.269301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f5f20-f268-4d93-95b5-8f5481a51c6d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.320777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.322185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.324513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.324513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.324513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.328372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.333468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 188/368] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:23.334210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.371615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.391883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7f489-c336-4fd7-b308-49ebfce0d83c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.437765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.438894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.439381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.442215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.460968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.467820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fe40f-435c-429f-abdf-7daf1a94cdde None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.519747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.520301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.520582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.520859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.521654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] RBAC: Adding query filter params (role.id=1c275785f4c947c994d1a4ef788661d3, user.id=41672b50df4e4542a3d6ff5ee36ebd61, scope.system=all) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:23.521966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.523454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.535808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.537717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.540164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 189/369] 10.222.0.85 () {56 vars in 1308 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/role_assignments?user.id=41672b50df4e4542a3d6ff5ee36ebd61&scope.system=all&role.id=1c275785f4c947c994d1a4ef788661d3 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:23.546105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.660933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.683551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4366d12-212f-46ba-8c61-483931612ea8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.732335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.733043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.733442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.733727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.748211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:23.754726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:23.755755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:23.756535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 190/370] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:23.757786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.798318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 181/371] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:23.799159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:23.909111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:23.918956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f2db2-bebf-4f04-8f6c-3515f8ebced0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:23.978321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:23.979974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:23.980356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:23.980641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:23.981457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:23.981769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:23.983282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:23.989661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 191/372] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:23.990663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.001815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f0092-f98f-4c77-9660-b283f63b4d4a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.007057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.063330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.063881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.064164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.064442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.065625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:24.065988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.068763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.081289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 182/373] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:24.089099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.130641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daacc2-18d7-4ee5-85a5-04c36b4996c8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.175517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.177171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.177683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.178195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.179814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.181526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.184212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 192/374] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:23 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:24.192926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.300095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.328654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24a57-3024-4d70-9ffb-ac75e97ad08e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.328654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.341611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee212c-15e0-436c-9e2d-194cfd3309a1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.361469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.362258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.362605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.362889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.398595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.399625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.400132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.400637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.402320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.405552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b83a8-13b2-4272-8ae5-fa53524f3a46 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.407996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 193/375] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:24.409522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadd065-be84-4eca-bd85-2873c435ed20 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.410017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadd065-be84-4eca-bd85-2873c435ed20 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.410530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadd065-be84-4eca-bd85-2873c435ed20 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.412042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 194/376] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:24.415282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadd065-be84-4eca-bd85-2873c435ed20 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.460888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.462457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e1b2a-b9e7-46e8-b635-3488d5a95965 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.538535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadd065-be84-4eca-bd85-2873c435ed20 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.581322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.582207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.582533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.582818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.583628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:24.583947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.585415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76958920-0ba9-4395-8cdb-8bde452f1d9c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.591832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 195/377] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:24.593073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce8465-10d8-47e5-99af-a56f15f6f94a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.593368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce8465-10d8-47e5-99af-a56f15f6f94a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.593654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce8465-10d8-47e5-99af-a56f15f6f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.595966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 183/378] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:24.597997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0cad3-7b89-4314-8755-5bb4bb4a5eb5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.598782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0cad3-7b89-4314-8755-5bb4bb4a5eb5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.599282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0cad3-7b89-4314-8755-5bb4bb4a5eb5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.601501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 184/379] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:24.602589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0cad3-7b89-4314-8755-5bb4bb4a5eb5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.613444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.615895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-59ce8465-10d8-47e5-99af-a56f15f6f94a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:24.617241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-59ce8465-10d8-47e5-99af-a56f15f6f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:24.620293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.732176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 196/380] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:24.733946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.734241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.734569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.755954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:24.757335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:24.768683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0cad3-7b89-4314-8755-5bb4bb4a5eb5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.808631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:24.809563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:24.809992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:24.810454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] PATH_INFO: `/v3/projects/eaffcb498d9c405ca0d18c97b43881f2` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:24.811816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=eaffcb498d9c405ca0d18c97b43881f2)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:24.813445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:24.815806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 185/381] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/projects/eaffcb498d9c405ca0d18c97b43881f2 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:24.816762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.888888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 197/382] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:24.890876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:24.938103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:24.974338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a99a-bd76-4167-a9b2-3784aef26eb9 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:24.982254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.026080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.026762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.027070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.027331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.029709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71773-a1b5-4238-90bf-84ac14374406 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.046114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:25.052414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:25.053404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:25.054198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 186/383] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:25.054891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.083224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.084471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.084950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.085627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] PATH_INFO: `/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.093014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] RBAC: Authorizing `identity:get_group(group_id=6cbcbab0a6ef46ecb3adc1b041f567d5)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.095514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.097777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 198/384] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:24 2022] GET /identity/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.099050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.218018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e3ca-cc45-45e5-bcde-c0a9edb2ce69 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.254668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9a96-66f4-455e-9608-2a8fb202822e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.270005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.271073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.272122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.272615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.275220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.293552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:25.302323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:25.305162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:25.306270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:25.307254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:25.310840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 187/385] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:25.312014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.313037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.313679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.314178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.314503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.315756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:25.316143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.317819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.331696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 199/386] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.332930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.471136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28973c34-473e-45ea-9da3-08b9732b16e6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.474450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3959c2-07ea-411c-8de9-1f4c79b8631b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.527669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.527669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.527669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.528807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.529917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] RBAC: Adding query filter params (group.id=6cbcbab0a6ef46ecb3adc1b041f567d5, role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=eaffcb498d9c405ca0d18c97b43881f2) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:25.530236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.531044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.536182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.538666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.550986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 188/387] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/role_assignments?group.id=6cbcbab0a6ef46ecb3adc1b041f567d5&scope.project.id=eaffcb498d9c405ca0d18c97b43881f2&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.551687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 200/388] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.553497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.558658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.650814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.654486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.714506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519353-9c7b-4d4c-9a1e-ea6984525876 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.742979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2cd4-519a-41a8-b5d6-785a42a95a7b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.764947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.765629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.765940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.766213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.767465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.769043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.771189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 189/389] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.779876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.817266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.818736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.819457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.820298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.823354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.827968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.832030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 201/390] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.847562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.912568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cfb42-1163-4c37-80f4-4d47f272e48a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:25.942450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:25.952668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:25.953492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:25.953832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:25.956843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:25.956843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:25.957525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:25.960631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 190/391] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:25.968016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:25.974296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2dd09-456b-4b53-824b-82fc09b108d2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:26.006875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:26.031453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:26.032121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:26.032443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:26.032698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:26.035463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:26.037053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:26.039022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 202/392] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:25 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:26.046231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:26.123184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cedce-5953-484a-92eb-c705ca61e9a4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:26.177267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:26.179882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:26.180858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:26.182618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/groups/7367a07df5a74944af9742f2197e8466/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:26.187022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, group_id=7367a07df5a74944af9742f2197e8466, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:26.190292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0f310-0340-4eff-9aef-dde6fa0b0d6f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:26.202401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb752d-32bd-438c-9998-162e1c1bee7c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:26.244298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:26.245863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:26.246420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:26.246903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf/groups/7367a07df5a74944af9742f2197e8466/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:26.249498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ac5424310548079b28ca6f050683cf, group_id=7367a07df5a74944af9742f2197e8466, role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:26.251185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ff986-375f-4318-9887-b1a75ec97d02 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:26.298703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 191/393] 10.222.0.85 () {58 vars in 1442 bytes} [Thu Jul 28 17:41:25 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/groups/7367a07df5a74944af9742f2197e8466/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:41:26.336819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 203/394] 10.222.0.85 () {58 vars in 1442 bytes} [Thu Jul 28 17:41:26 2022] PUT /identity/v3/projects/28ac5424310548079b28ca6f050683cf/groups/7367a07df5a74944af9742f2197e8466/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:26.705594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd766a7-71b0-488b-b4f8-5766ac837cc8 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:26.706253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd766a7-71b0-488b-b4f8-5766ac837cc8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:26.706834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd766a7-71b0-488b-b4f8-5766ac837cc8 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:26.708656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 192/395] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:26.714965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:26.715687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:26.716167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:26.741154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:26.742713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:26.810750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:26.835535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 204/396] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:26.872324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd766a7-71b0-488b-b4f8-5766ac837cc8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:26.955466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:27.005687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd766a7-71b0-488b-b4f8-5766ac837cc8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:27.046240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:27.046933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.047219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.047500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.063643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:27.069199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:27.070111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:27.070946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 193/397] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:27.076620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.187040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b5e1-7147-4fc8-889f-fa4a9f11f7d6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:27.243075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173b485-8e12-4585-b811-2632862005df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:27.243720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173b485-8e12-4585-b811-2632862005df None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.244006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173b485-8e12-4585-b811-2632862005df None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.244286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173b485-8e12-4585-b811-2632862005df None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.245649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173b485-8e12-4585-b811-2632862005df None admin] RBAC: Adding query filter params (name=member) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:27.245964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173b485-8e12-4585-b811-2632862005df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:27.247638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173b485-8e12-4585-b811-2632862005df None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:27.257691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 205/398] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:27.263366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.339983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7bb4e-91bc-4c73-944a-a98ecfbb431b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.340336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7bb4e-91bc-4c73-944a-a98ecfbb431b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.340597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7bb4e-91bc-4c73-944a-a98ecfbb431b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.343161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 206/399] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:27.347473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.347756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.348107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.383485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:27.384763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:27.410781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767444c-fd37-4cf5-bbf7-07b3260ef654 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:27.430394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:27.453220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:27.453967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.454257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.454565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.456073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:27.457799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:27.461243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 194/400] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:27 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:27.467156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.495844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:27.544361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 207/401] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:27.578911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.595928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd07f-0993-4216-9fc2-f1096b0797a6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:27.665827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:27.667484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.667869 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.668154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.671863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:27.674910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:27.681081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 195/402] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:27 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:27.686777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.766645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771077d-ef6a-4708-827d-83de6c7209ac None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:27.809083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:27.809684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:27.809971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:27.810242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:27.822813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:27.829110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:27.830052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:27.830901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:27.831546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 208/403] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:27.839912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:27.859863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:27.904635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:27.958259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb6be4-20b9-4cc8-b587-9a714a8e3f23 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.033789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.035067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.036801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.038129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/groups/7367a07df5a74944af9742f2197e8466/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.041217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, group_id=7367a07df5a74944af9742f2197e8466, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:28.043523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:28.053022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d564e-3d9e-46ac-940b-2e9682604dc0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.099575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.100336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.100526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.100769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.101511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:28.103651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:28.106119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:28.119040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 209/404] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:28.125198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:28.164781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 196/405] 10.222.0.85 () {58 vars in 1442 bytes} [Thu Jul 28 17:41:27 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/groups/7367a07df5a74944af9742f2197e8466/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:28.262752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:28.269340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b682b-bc0e-4a81-9528-2d44d40c05c7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.311897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.312565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.312857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.313167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.314429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:28.316099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866feb7-106a-45aa-9afb-d5f5f81047c8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:28.318146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 210/406] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:28 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:28.324402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:28.336818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1b65-d1a7-4560-9b54-17f3d48d5ffa None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.337111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1b65-d1a7-4560-9b54-17f3d48d5ffa None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.337415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1b65-d1a7-4560-9b54-17f3d48d5ffa None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.339772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 211/407] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:28.354792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.354792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.354792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.382637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:28.383872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:28.478198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2435b-b5ab-40d0-9602-044cb77e20dd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.490252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:28.519848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.521271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.521562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.521888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.523696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:28.525391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:28.530297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 197/408] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:28 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:28.535842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 212/409] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 28 17:41:28.536840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:28.560884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:28.681233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d0ad-cdb7-4e9c-8915-0534e9e7b606 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.717319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.718005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.718290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.718697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/groups/7367a07df5a74944af9742f2197e8466/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.720990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae, group_id=7367a07df5a74944af9742f2197e8466, role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:28.722600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:28.735945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:28.743189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708915-ffac-41e5-ba74-d6df95040f60 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:28.779807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:28.780465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:28.780749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:28.781029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:28.792887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:28.798381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:28.799269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:28.800061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 213/410] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:28.809552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 198/411] 10.222.0.85 () {58 vars in 1442 bytes} [Thu Jul 28 17:41:28 2022] PUT /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae/groups/7367a07df5a74944af9742f2197e8466/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:28.810423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:28.967100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc522e-dc36-4c7f-b287-3f64f0b46df2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:29.015872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399726-fd59-41e4-9159-690842157331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:29.016501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399726-fd59-41e4-9159-690842157331 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.016789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399726-fd59-41e4-9159-690842157331 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.017066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399726-fd59-41e4-9159-690842157331 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.017793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399726-fd59-41e4-9159-690842157331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:29.018099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399726-fd59-41e4-9159-690842157331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:29.019526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399726-fd59-41e4-9159-690842157331 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:29.029639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 214/412] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:29.036891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.084217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:29.155252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa568dc-8209-4694-b878-2a9fbfda15c9 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:29.193082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:29.193741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.194028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.194310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] PATH_INFO: `/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.195573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] RBAC: Authorizing `identity:get_group(group_id=6cbcbab0a6ef46ecb3adc1b041f567d5)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:29.197033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376c3e4-05d2-4266-a0a4-6a3f17643538 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:29.200078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 199/413] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:29 2022] GET /identity/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:29.206438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399726-fd59-41e4-9159-690842157331 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.231456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:29.348843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399726-fd59-41e4-9159-690842157331 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:29.375702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ac27-9c61-4f88-bec6-d6f5ad38944e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.376032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ac27-9c61-4f88-bec6-d6f5ad38944e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.376254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ac27-9c61-4f88-bec6-d6f5ad38944e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.377760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 200/414] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:29.383669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2356e32-e880-49e7-8038-c2d309e3ae43 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.384161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2356e32-e880-49e7-8038-c2d309e3ae43 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.384645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2356e32-e880-49e7-8038-c2d309e3ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.410939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:29.411642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.411927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.412246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] PATH_INFO: `/v3/projects/eaffcb498d9c405ca0d18c97b43881f2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.415110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] RBAC: Authorizing `identity:get_project(project_id=eaffcb498d9c405ca0d18c97b43881f2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:29.417266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8637e5-4ebf-4ae3-b6f9-e045ac15967a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:29.423110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 215/415] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:29 2022] GET /identity/v3/projects/eaffcb498d9c405ca0d18c97b43881f2 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:29.425298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e932101-6ea4-4f20-a131-f78ab89f99e3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.428278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e932101-6ea4-4f20-a131-f78ab89f99e3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.428490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b2356e32-e880-49e7-8038-c2d309e3ae43 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:29.428653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e932101-6ea4-4f20-a131-f78ab89f99e3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.428804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b2356e32-e880-49e7-8038-c2d309e3ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:29.428953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 216/416] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:29.428953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e932101-6ea4-4f20-a131-f78ab89f99e3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.510672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 201/417] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:29.512326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.512618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.512901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.544177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:29.546010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:29.550707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:29.572460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e932101-6ea4-4f20-a131-f78ab89f99e3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:29.583329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:29.612815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:29.613897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.614435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.614951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] PATH_INFO: `/v3/projects/eaffcb498d9c405ca0d18c97b43881f2/groups/6cbcbab0a6ef46ecb3adc1b041f567d5/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.617333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] RBAC: Authorizing `identity:create_grant(project_id=eaffcb498d9c405ca0d18c97b43881f2, group_id=6cbcbab0a6ef46ecb3adc1b041f567d5, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:29.619097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:29.634422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 202/418] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:29.642455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.702592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 217/419] 10.222.0.85 () {58 vars in 1442 bytes} [Thu Jul 28 17:41:29 2022] PUT /identity/v3/projects/eaffcb498d9c405ca0d18c97b43881f2/groups/6cbcbab0a6ef46ecb3adc1b041f567d5/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:29.704172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.756458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ca994-3eb4-439d-b40a-27f2c5c70630 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:29.829788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:29.831572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:29.841753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:29.842064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:29.862775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:29.873831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:29.874809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:29.875677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:29.876740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 203/420] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:29.882603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:29.948270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:29.960404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c32dd9-17d4-4aa3-b8b7-c7056b2fc861 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.001968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.003515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.003994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.004490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.016657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:30.022618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:30.023476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:30.024284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:30.027894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 218/421] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:30.039706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.058769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.074178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0635a-a1a4-4170-a92e-d6c9443230fe None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.112222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.112814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.113101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.113378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.114147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:30.114494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.115933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.140436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 204/422] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:30.146184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.242845 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.255917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cd54d-a1fa-4d29-af23-5483e3ff6284 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.264353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d149d-8eba-4bfa-ae67-2a6d8a824f83 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.296331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.296968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.297256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.297535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.372945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.374392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.416294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.417117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.417406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.417690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.418583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:30.418894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.420556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.451885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 219/423] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:30.459899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.510903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.540682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 205/424] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:41:30.546873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.594724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.642051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaddf1-826c-4e43-9f5c-f1d6d36debdc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.679998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.680721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.681011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.681291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.704631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97237fa4-377e-4906-920a-1ed6fe2d5058 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.743208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.743851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.744137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.744951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.746312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.747787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9603b-3a00-4865-9d03-ad2d5cd76863 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.749740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 206/425] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:30.751009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7a9-cd49-4510-a7e0-bc5dc0c20714 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.751295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7a9-cd49-4510-a7e0-bc5dc0c20714 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.751572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7a9-cd49-4510-a7e0-bc5dc0c20714 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.753099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 207/426] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:30.756041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb7a9-cd49-4510-a7e0-bc5dc0c20714 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.766799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.768448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce54f-a4be-416b-845d-10ff7e56dd51 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.817179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.843237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:30.873838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb7a9-cd49-4510-a7e0-bc5dc0c20714 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:30.885149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 220/427] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:30.887665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.888126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.888580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.909206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:30.910778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:30.915632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:30.916854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:30.917297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:30.917600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:30.918833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:30.920734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:30.922671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 208/428] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:30.923520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:30.990186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 221/429] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:30.998371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.042328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4427d6-fd02-4ed5-a81e-b1bafc110da3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.067947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.080313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.080963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.081253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.081529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] PATH_INFO: `/v3/projects/28ac5424310548079b28ca6f050683cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.082868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=28ac5424310548079b28ca6f050683cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.084429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.092474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 209/430] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/projects/28ac5424310548079b28ca6f050683cf => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:31.093084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.113319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.121119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f08b6-8c97-401a-abb4-1efd01f4d5f1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.160189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.161237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.161788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.162239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.163246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=28ac5424310548079b28ca6f050683cf) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:31.163739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.165548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.173970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 222/431] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:30 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:31.175106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.211143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f871a12-0ebf-4fbc-bafe-8a89bd329bd1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.253167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.253895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.254220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.255035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.267322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:31.272704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:31.273614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:31.274428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 210/432] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:31.287596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.297703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ba055-41cc-4401-abd1-bdeaee773a85 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.337363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.339396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.339932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.340404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.341952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.343676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.345852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 223/433] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:31.351908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.383781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.384420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.470975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb84e-ec9f-44e7-9ff6-54b1891bf2a5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.478199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077af0-7bbd-4021-ad1d-0164af899c4a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.516428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.517718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.518212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.518869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.520378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=28ac5424310548079b28ca6f050683cf) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:31.521139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.523271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5d95-95e9-4f13-9cd1-f12ecc111ed2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.532210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 224/434] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=28ac5424310548079b28ca6f050683cf&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:31.532706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.533335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.533620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.533893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.534005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fd3-f557-4eb2-a1a0-0d9bf965e31c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.534324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fd3-f557-4eb2-a1a0-0d9bf965e31c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.534660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:31.534968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.536356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.538277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fd3-f557-4eb2-a1a0-0d9bf965e31c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.540308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 225/435] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:31.552049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 211/436] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:31.553558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.553875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.554157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.558627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.583520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:31.584902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:31.674547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 226/437] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:31.698637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.708079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4a5-8296-4e5b-9b95-7b0f738a3cb8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.725562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.729535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:31.748926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.749552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.760809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.760809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.828976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf2774-1871-4d65-b794-a1d1a55e4e86 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:31.846757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:31.848340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:31.874838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:31.875523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:31.875809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:31.876090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:31.889799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:31.894997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:31.895916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:31.898966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 227/438] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:31.903410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:31.992059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 212/439] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:31.999008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.018334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.020568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.049874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0ca5d-70db-49a3-ab90-e5b8839b1d88 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.095202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.095850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.096137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.096415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.097468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:32.097468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.099163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4cc2-c651-4148-a3ad-b49990578fcc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.111287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 228/440] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.111566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb3da1-406c-44ff-a9e2-7c9a54347129 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.111847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb3da1-406c-44ff-a9e2-7c9a54347129 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.112129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb3da1-406c-44ff-a9e2-7c9a54347129 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.113550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 229/441] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:32.120809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f67c-254d-4692-821c-c25f4291179c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.121218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f67c-254d-4692-821c-c25f4291179c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.121503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438f67c-254d-4692-821c-c25f4291179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.133816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c597-c070-46d0-a41d-06e12e3ea95e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.141661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c438f67c-254d-4692-821c-c25f4291179c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:32.142991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c438f67c-254d-4692-821c-c25f4291179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:32.177745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.178512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.178866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.179217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.180390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.182394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.184121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 213/442] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:31 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.184894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.222696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 230/443] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:32.223615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438f67c-254d-4692-821c-c25f4291179c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.279049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.297090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041e75-855c-43ed-988e-24bd419e30d8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.307683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.332025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.332650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.332940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.333223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.333327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438f67c-254d-4692-821c-c25f4291179c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.369777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.370497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.370785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.371066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.372234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.373750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.375392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 231/444] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.376163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.405117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.406590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.477877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bfd7b-ff1b-410a-9629-7a68734bb666 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.488380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.507377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 214/445] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:32.508218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.515497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.516144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.516428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.516713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.528962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:32.534340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:32.535290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:32.536203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 232/446] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:32.537377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.539684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.616167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e9341-9861-4347-af32-7a4378a4016d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.639111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad411cd-ad68-45b2-96cb-e44a771a7e8c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.651462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.652084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.652369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.652654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.653463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=ec41984b84ff479d8a0e7ad9405aa8b1, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:32.653774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.655245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.661080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 215/447] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=ec41984b84ff479d8a0e7ad9405aa8b1 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.661881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.684453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.685127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.685416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.685698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] PATH_INFO: `/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.687101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] RBAC: Authorizing `identity:get_project(project_id=2e58a7d8ccbc49ddabf18e1cfc4b27ae)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.688651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.691408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 233/448] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/projects/2e58a7d8ccbc49ddabf18e1cfc4b27ae => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.697029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.726372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.782636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.790247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba9cbf-8b55-4e64-9fdd-4d0dfbd5083a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.818778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3acf4-05cd-4430-9a45-32970e2b4d8e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.836602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.836602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.836602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.836602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.837001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:32.837001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.838963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.850557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 216/449] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:32.855219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.860753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:32.861402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:32.861690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:32.861971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] PATH_INFO: `/v3/groups/7367a07df5a74944af9742f2197e8466` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:32.863186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] RBAC: Authorizing `identity:get_group(group_id=7367a07df5a74944af9742f2197e8466)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:32.864638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:32.866428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 234/450] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/groups/7367a07df5a74944af9742f2197e8466 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:32.872935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:32.971218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af36f2-0696-478c-a016-9a50757b5fff None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:32.981744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:32.994429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b197fe-35ee-4bb2-85e7-f37ca4a66993 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:33.014979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:33.015450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:33.016323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:33.016323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:33.038929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:33.040161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:33.040161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:33.040161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:33.040965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] RBAC: Adding query filter params (group.id=7367a07df5a74944af9742f2197e8466, role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:33.041821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:33.043576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:33.049942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 235/451] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/role_assignments?group.id=7367a07df5a74944af9742f2197e8466&scope.project.id=2e58a7d8ccbc49ddabf18e1cfc4b27ae&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:33.093513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:33.095520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:33.207744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 217/452] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:33.214164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:33.217099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:33.321682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad073f-5edd-4049-bd4c-f7f8d45d6747 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:33.357364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:33.358016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:33.358306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:33.358615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] PATH_INFO: `/v3/projects/eaffcb498d9c405ca0d18c97b43881f2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:33.359790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] RBAC: Authorizing `identity:get_project(project_id=eaffcb498d9c405ca0d18c97b43881f2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:33.361274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:33.363338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 236/453] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:41:33 2022] GET /identity/v3/projects/eaffcb498d9c405ca0d18c97b43881f2 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:33.368632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:33.380347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:33.471999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95786c-3532-4c51-b2f1-c6525729c2a6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:33.507257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:33.507878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:33.508157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:33.508430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] PATH_INFO: `/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:33.509556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] RBAC: Authorizing `identity:get_group(group_id=6cbcbab0a6ef46ecb3adc1b041f567d5)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:33.511063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b04b2-ec48-44f0-bf96-c2d390be181f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:33.512636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 218/454] 10.222.0.85 () {56 vars in 1180 bytes} [Thu Jul 28 17:41:33 2022] GET /identity/v3/groups/6cbcbab0a6ef46ecb3adc1b041f567d5 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:33.518561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:33.587338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:33.618240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ba1b-56be-4220-92cc-5defeaae0299 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:33.649923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:33.651119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:33.651407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:33.651784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:33.652571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] RBAC: Adding query filter params (group.id=6cbcbab0a6ef46ecb3adc1b041f567d5, role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=eaffcb498d9c405ca0d18c97b43881f2) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:33.652913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:33.654367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83d7cc-a2bf-497a-b025-37c985d9712c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:33.660066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 237/455] 10.222.0.85 () {56 vars in 1376 bytes} [Thu Jul 28 17:41:33 2022] GET /identity/v3/role_assignments?group.id=6cbcbab0a6ef46ecb3adc1b041f567d5&scope.project.id=eaffcb498d9c405ca0d18c97b43881f2&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:36.184040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4dfc-2249-4179-98bd-5a7aa103feba None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.184782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4dfc-2249-4179-98bd-5a7aa103feba None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.184782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4dfc-2249-4179-98bd-5a7aa103feba None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.186223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 219/456] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:36.194470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.195732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.196451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.235708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:36.237954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:36.393664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a5e6-0974-4f57-ab6b-c5aa9d13ee68 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.393995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a5e6-0974-4f57-ab6b-c5aa9d13ee68 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.394229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a5e6-0974-4f57-ab6b-c5aa9d13ee68 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.395924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 220/457] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:36.399280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 238/458] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:36.417831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.418166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.418417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.443482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:36.456452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:36.457705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:36.564051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 221/459] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:36.588933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:36.617010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:36.628421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:36.666916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb62b3-38a1-4e97-978a-a77b5095a115 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:36.728196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:36.729518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.729897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.730184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.754732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:36.756250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:36.803206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:36.804835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:36.806426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:36.806426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:36.806426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:36.806426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:36.806426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 239/460] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:36.813403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:36.813691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9199-780d-450b-aeef-e6b91ad67499 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:36.861578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:36.862267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.862584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.862867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.877205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:36.879395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1f1ea84c-761a-4651-a43a-79c062e3d2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:36.898145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:36.899779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 222/461] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31892b2-dd34-40b6-b607-9fd7660810ce None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31892b2-dd34-40b6-b607-9fd7660810ce None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:36.901839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31892b2-dd34-40b6-b607-9fd7660810ce None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:36.907377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 223/462] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:36.909187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31892b2-dd34-40b6-b607-9fd7660810ce None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:36.985554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5c79c-7d58-4984-9434-326d8d9572a4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:36.993749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:37.006670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:37.028417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.029190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.029476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.029755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.030494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:37.030807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.033764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292e79d-fa53-4826-a513-c4c7d1c27112 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.051891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 240/463] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:37.053585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.054309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.054815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.055451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31892b2-dd34-40b6-b607-9fd7660810ce None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:37.075267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:37.076889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:37.100351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.101534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.101828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.102117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.102861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:37.103170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.105722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.130377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 224/464] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:37.130377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:37.168546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 241/465] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:37.170159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:37.293765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347eb97-491a-4fb2-811f-cfca61c3ad07 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:37.299885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d83a2a-7a48-48dd-8b18-5d6879f6e6ee None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:37.309908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:37.317665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:37.341678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.342662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.342957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.343083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.343269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.343966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.344418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.344893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.345498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.345741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.346672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.348835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.458382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 242/466] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:37 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:37.458382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:37.523587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 225/467] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:37 2022] POST /identity/v3/users => generated 246 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:37.633876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68278c8a-3db7-4aeb-806a-8b664e4d255f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:37.698506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.698506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.698506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.699149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.712363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.714834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-25832ce1-5d84-47b0-aed0-4d35834a2296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:37.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:37.732561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:37.733449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:37.733449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:37.733449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:37.733449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:37.734022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 243/468] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:37.740303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:37.747351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f94bd-31ed-4886-9b07-e04c9bbb4a28 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.747629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f94bd-31ed-4886-9b07-e04c9bbb4a28 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.747912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f94bd-31ed-4886-9b07-e04c9bbb4a28 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.749781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 244/469] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:37.755611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.755897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.756487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.781441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:37.782692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:37.802267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:37.898835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6de794-fec7-495a-9d6d-94ca7399294f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:37.905184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 245/470] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:37.933436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:37.944336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:37.945085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:37.945437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:37.945811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:37.946614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:37.946987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:37.949002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:37.962098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 226/471] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:37.968426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.012537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.040702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d8141-5aa2-477a-9c00-009f14f7e926 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.079190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.079855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.080138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.080418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.088744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5b48a-173e-425e-b524-48688abd11fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.091721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:38.093221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:38.096059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8a956ace-4769-4964-b1f2-885649cac277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:38.109408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:38.110746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:38.111793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:38.111793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:38.111793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:38.111793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:38.114465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 246/472] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:38.115756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd206b-52c3-4414-9390-4b6a8a0d7340 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.116048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd206b-52c3-4414-9390-4b6a8a0d7340 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.116331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd206b-52c3-4414-9390-4b6a8a0d7340 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.117680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 247/473] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:38.120814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd206b-52c3-4414-9390-4b6a8a0d7340 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.139852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.140951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.141439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.141995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.143477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:38.145260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4384c-02c7-4f0d-bac5-5d5e4ff69daa None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:38.231576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 227/474] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:37 2022] POST /identity/v3/users => generated 248 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:41:38.234395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.235009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.235779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.245265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd206b-52c3-4414-9390-4b6a8a0d7340 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.265543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:38.267461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:38.338900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.339881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.340197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.340620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.341774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:38.342238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:38.344369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:38.374191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 248/475] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:38.380596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.385889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 228/476] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:38.431248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.467249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.471612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.520408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc7601-9deb-4751-b0d1-7f79fb257313 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.563005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.563639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.563925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.564207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.564999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:38.566484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:38.575022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762952a-6cbe-4a4b-babb-670128def3e5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.622019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.623039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.623323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.623598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.624333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:38.625827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:38.646752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 249/477] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:38 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:41:38.660761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.sql.core [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:41:38.660761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:41:38.660761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [parameters: {'id': '1fde249901094b9bb3055fbe4ccaf2b6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:41:38.660761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92084) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:41:38.679561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:41:38.680690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application yield resource Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise self.value Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application session.commit() Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:41:38.682485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:41:38.683537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:41:38.684644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:41:38.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [parameters: {'id': '1fde249901094b9bb3055fbe4ccaf2b6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:38.686786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:41:38.687784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:38.692309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 229/478] 10.222.0.85 () {60 vars in 1131 bytes} [Thu Jul 28 17:41:38 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:38.695531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.815483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a451-eea3-4df3-b250-0678fa037f4c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:38.833774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.852832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:38.855269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:38.855269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:38.855269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:38.867150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f1f961df-e81a-45dc-bf3c-e254617df774 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:38.873767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:38.885829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:38.887043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 250/479] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:38.887948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:38.979327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:38.996886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7a0d-3d00-48c7-8d4f-1e2dbc974e76 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:39.035263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:39.035902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:39.036188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:39.036483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:39.037215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:39.037773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:39.039233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06bccd-be5b-4f0a-a88e-59b7f2a2be07 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:39.054896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 230/480] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:39.975188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3019-e858-4143-b689-5a65e7e88616 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:39.975691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3019-e858-4143-b689-5a65e7e88616 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:39.975864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3019-e858-4143-b689-5a65e7e88616 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:39.977362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 251/481] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:39.983349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d3ab-0b56-43c5-92cb-018ebe8327ab None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:39.983801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d3ab-0b56-43c5-92cb-018ebe8327ab None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:39.983981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d3ab-0b56-43c5-92cb-018ebe8327ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.004125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3693d3ab-0b56-43c5-92cb-018ebe8327ab None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:40.005343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3693d3ab-0b56-43c5-92cb-018ebe8327ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:40.101007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 231/482] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:40.133973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3019-e858-4143-b689-5a65e7e88616 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:40.143574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0970d77-c076-4ca6-91a9-d4ac9c3f8af6 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.143873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0970d77-c076-4ca6-91a9-d4ac9c3f8af6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.144155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0970d77-c076-4ca6-91a9-d4ac9c3f8af6 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.145641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 232/483] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:40.151257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.151857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.152362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.188431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:40.193400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:40.319690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 233/484] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:40.339089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3019-e858-4143-b689-5a65e7e88616 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:40.345915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:40.355442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:40.422284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:40.422284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.422284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.422284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.443909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:40.449775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:40.450707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:40.451502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:40.452102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 252/485] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:40.459491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:40.507173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6d1a-bcb3-44b4-80aa-99354bb6f248 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:40.524994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:40.555034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:40.556102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.556623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.557995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.598013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:40.609434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:40.610323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:40.611175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:40.617145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 234/486] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:40.636245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:40.653269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235c5fa-0cad-43ed-9119-5fb2eb8601f7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:40.743380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:40.755597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:40.756233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.756515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.756792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.757522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:40.757828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:40.759253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:40.778853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 253/487] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:40.784063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:40.800441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e23280-1dcf-4a3b-9875-cb6da54a5e2e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:40.870052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:40.870639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.870931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.871217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.871952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:40.872268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:40.873771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed6e7e-b619-4aae-baaf-4a129fa14228 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:40.887130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 235/488] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:40.888099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:40.888528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529a9c-e012-4817-90a6-0b9cd4367131 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.888828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529a9c-e012-4817-90a6-0b9cd4367131 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.889156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529a9c-e012-4817-90a6-0b9cd4367131 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.890823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 236/489] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:40.893459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda286-4695-467f-a9be-398024cc09e3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:40.899471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.899471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.899471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.918195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:40.919724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:40.938203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:40.938928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:40.939215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:40.939494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:40.957929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:40.960224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:40.982908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:40.984366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:40.985384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:40.985384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:40.985384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:40.985384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:40.987818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 254/490] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:40.988773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.014497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 237/491] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:41.015718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.099052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.111199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6dff-572a-40e2-b59b-b242b943a698 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.147921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.148825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.148938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd4497-9d7b-4d70-80fb-f579dfb71ea2 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.149343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.149761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.161986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.163814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.173748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:41.183873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:41.185812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 255/492] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:41.186769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.199502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.200504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.201033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.201569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.203926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:41.204511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.206163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.220997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 238/493] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:41.222476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.323831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da200a7-0503-45f0-9ec9-1bb53aba7787 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.366169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.367181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.367613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.368064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.368683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372dd70-fcdb-4e9f-a6f7-50974b236870 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.384291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:41.393425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:41.394598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:41.395590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:41.397063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.399254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 256/494] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:41.400294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.422533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.423501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.423929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.424378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.425596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:41.426249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.427986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.443477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 239/495] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:41.449890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.480071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.544892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cab93-1810-4ab0-9e24-15a94ae5fa7c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.599288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.599288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.599593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.599748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.601654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61da4-3be0-4a78-b418-80ac44c7447f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.608222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.609721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:41.625432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:41.626422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:41.627308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:41.630539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 257/496] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:41.632519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.663507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.664054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.664340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.664623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.665349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:41.665715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.667251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e507e-f8c5-4e83-b328-e4417d6c0c34 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 240/497] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98591-8ade-49fe-9a6a-42b7786c34ce None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98591-8ade-49fe-9a6a-42b7786c34ce None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98591-8ade-49fe-9a6a-42b7786c34ce None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 241/498] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:41.684349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da98591-8ade-49fe-9a6a-42b7786c34ce None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.741865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.778217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e49e61-1d13-4664-b81b-7987bddf09e0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.798809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:41.815736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da98591-8ade-49fe-9a6a-42b7786c34ce None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:41.821701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.823496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.823980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.824394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.839887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.841321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:41.853810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:41.854851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:41.855768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:41.858501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 258/499] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:41.860033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:41.866494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:41.867120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.867409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.867690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.868501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:41.869409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:41.871129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ee19b-5a13-4fd3-9e28-6b746e42dd94 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:41.912506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 242/500] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:41.913937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c42f-e02b-40dd-9cbd-ba3e2162dcb6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:41.914253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c42f-e02b-40dd-9cbd-ba3e2162dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:41.914570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c42f-e02b-40dd-9cbd-ba3e2162dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:41.936044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fa46c42f-e02b-40dd-9cbd-ba3e2162dcb6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:41.937606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fa46c42f-e02b-40dd-9cbd-ba3e2162dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:41.998674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12bf2d-680d-4bc1-8734-4d766ed09480 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.035639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 243/501] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:42.036798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ab503-eff9-4e55-92f7-3859fb7eb3ea None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.037092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ab503-eff9-4e55-92f7-3859fb7eb3ea None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.037374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ab503-eff9-4e55-92f7-3859fb7eb3ea None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.040095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 244/502] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:42.040095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ab503-eff9-4e55-92f7-3859fb7eb3ea None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.048565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.050045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.050554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.051010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.067222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.069836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.071751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:42.093846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:42.095010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 259/503] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:42.095989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.139614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.164815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ab503-eff9-4e55-92f7-3859fb7eb3ea None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.211918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.212602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.212899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.213184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.214007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:42.214323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.215839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda32149-8f50-41bd-9c8e-b8d57be489ef None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.231657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb23e7-5544-4ea7-8d7d-a55662a9c7f8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.242109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 245/504] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:42.243765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.244803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.244803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.267109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:42.268350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:42.274107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.274777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.275102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.275386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.341595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 246/505] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:42.342498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.348986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.350602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.369798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.383013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.461561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cf400-d351-4850-9694-90dc75cf5756 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.471114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 260/506] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:42.473988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.508380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.509026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.509363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.509640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.520240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:42.525271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:42.526309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 247/507] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:42.532252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.581632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99c61-21d2-4eea-9d7f-5f2dbed80512 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.609121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.610739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.616421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.622566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.630973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 261/508] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:42.631772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.631872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91a7f-cd31-42c2-be45-8050cd15f666 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.670032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.670600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.670884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.671160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.735209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76cba-dad3-4e2b-9156-74a41aa988b1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.740913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.742481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.769625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.770267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.770585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.770864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.781498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.783015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:42.797907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:42.798917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:42.799734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:42.799734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:42.799734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.799734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:42.799734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 262/509] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:42.800005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.824809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.850028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:42.851332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 248/510] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:42.851332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.909867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a446c-d3d4-4e19-9a34-1b6a6239fe25 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.945391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.946043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.946327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.946643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.952500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240da15-4759-4839-86c5-77ecfd535d30 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:42.958715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.960203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:42.975411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:42.976331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:42.977373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:42.977373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:42.977373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:42.977373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:42.977373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 263/511] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:42.977804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:42.991429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:42.991988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:42.992612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:42.992895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:42.993625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:42.994547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:42.995944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.004172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 249/512] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.007298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.048385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.076840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.083660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9fee9-3ddf-40df-9ffd-f0990230fa97 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.109789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbd5e6-be1c-4250-9111-c25c152382fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.119888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.120538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.120849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.121135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.130740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.132225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.145669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.146300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.146616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.146897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:43.147366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:43.148218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:43.149129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:43.149649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:43.149649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.149777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 264/513] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:43.149841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.149904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.162198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 250/514] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.163153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.265910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716afee-63d6-4719-bda2-81a4b55bc833 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.266544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.270814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610c53-1484-49b5-9e31-749b6630c0cd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.302938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.303587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.303874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.304160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.308640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.309257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.309545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.309822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.310554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:43.310870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.312246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.315548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.317011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.323474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 251/515] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.324308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:43.331575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:43.332441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:43.333268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:43.333268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:43.333268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:43.333268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:43.333268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 265/516] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:43.333866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.343357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.436334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb824b-a0be-4a43-a12b-d4b1fae233ed None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.445137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef9297-1754-41da-ad00-3d165cd9c0a8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.470900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.471537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.471821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.472095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.480124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.480847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.481135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.481452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.482279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:43.482637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.484077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.484176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.485618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:43.502675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:43.503807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 252/517] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:43.504674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.518390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.521598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 266/518] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.522618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.588853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.611885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77871e-3816-4552-829a-4ec2ebe9b9ba None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.629399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd815e-d35a-421d-80df-5e4b1c15af36 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.658919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.659551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.659840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.660990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.660990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.662290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.669915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.670589 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.670876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.671161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.678250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.679754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:43.693015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:43.694035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 267/519] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:43.694953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.745412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 253/520] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:41:43 2022] POST /identity/v3/users => generated 247 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:41:43.746321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.844626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.849782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365a23d-47d3-4c69-8ceb-44b94be2c19e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.880295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:43.894782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.895424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.895707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.895847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e6bd8-ef5d-4dae-89b9-d6bf88e8aaea None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:43.896163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.896689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:43.897776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.898515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.917476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 268/521] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.918748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:43.945009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:43.945650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:43.945933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:43.946706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:43.947497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:43.947814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:43.949224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:43.966649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 254/522] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:43.966649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.068618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7569d08-4741-43a4-9398-68e6a611fd8e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.104224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b714fd-0ce6-4096-a947-acac5c69faf6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.123950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.125327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.125327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.125327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.152411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.170402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.171241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.171589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.176173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.176173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:44.176173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.176173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.196747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 255/523] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:44.197928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.210471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.231878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.233719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.324637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7355-58f4-4a81-8275-28d7f2908e27 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.353949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 269/524] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:44.355468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.367848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.368510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.369074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.369074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.376427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.377908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:44.390396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:44.391298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:44.392117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 256/525] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:44.392569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.450092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.465293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.482289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d78a2-dd6c-419d-9679-2bfd91f5fc2d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.514366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205dc87-72a3-475b-9d45-4d4a3cd7e930 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.522597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.523318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.523673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.524022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.532339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.533832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:44.546074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:44.547302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 270/526] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:44.548200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.555798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.556464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.556734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.557014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.557824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:44.558142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.559996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.578558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 257/527] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:44.579566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.671630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0050c-5fea-49ff-9da2-5410cc26aae3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.704766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d777a-d2e8-4dc5-ac27-f65f94d2d412 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.710988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.711655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.711945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.712229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.719601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.725905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.730104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.731816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:44.745804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:44.746899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:44.747941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:44.747941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:44.747941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:44.747941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:44.752754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 271/528] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:44.753668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.760554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.764598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.764598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.764598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.764598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:44.764598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.765248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.779921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 258/529] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:44.781031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.867143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d1da3-2685-4c45-8812-d2efc0cf511f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.896272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04126ca9-f1e5-4701-abd8-2bb6492f5e1b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:44.905867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.906790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.907117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.907495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.908453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:44.908866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.910487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.932629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 272/530] 10.222.0.85 () {56 vars in 1139 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:44.937704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:44.943443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:44.944390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:44.944887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:44.945360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:44.946250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:44.947556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:44.949201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:44.966822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 259/531] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:44.968116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:45.001598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.034839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.053402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e15b6-d0be-4bdd-a712-36f39686581d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.079729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eb4c7-8cb9-46aa-8e63-24b66eb5cc49 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.092980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.093637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.093927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.094210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.095046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=9e8966ba17c04e1782a6ed9fb4e59254) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:45.095371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.096815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.104041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 273/532] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/role_assignments?user.id=9e8966ba17c04e1782a6ed9fb4e59254&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:45.104921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:45.120333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.121116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.121403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.121682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.227723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.232524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.281096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef208c-b0da-49a1-b94c-7de274b3bf78 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.298729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.342947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.387682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.388623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.389188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.389617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.408123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.409879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:45.431628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:45.432725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:45.433778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:45.440151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 274/533] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:45.445329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:45.447425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 260/534] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:45.459393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:45.576787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961a83d-c4a4-4935-b816-61f2138d3984 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.615005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.615733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.616022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.616301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.617089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:45.617368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.618900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.638342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 275/535] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:45.644230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:45.691278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.714930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ee64-177c-4d9c-b9ce-329e0af86fc8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.765283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e645-9a98-4b94-ac8f-6a7198c7d895 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:45.800396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:45.804291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.804918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.805199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.805516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.806878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=4573990adc20477ab369cd717aaf0679) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:45.807178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.808704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.815803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 276/536] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:45 2022] GET /identity/v3/role_assignments?user.id=4573990adc20477ab369cd717aaf0679&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:45.816103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:45.816851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:45.817147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:45.817428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:45.855527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:45.857316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:45.883807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:45.884939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:45.885944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:45.885944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:45.885944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:45.885944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:45.892309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 261/537] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:45.902769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.052231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1952852-aced-4e4e-815f-153e499f30fe None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.053824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:46.094384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.095113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.095398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.095677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.096496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:46.096809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:46.098196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d29f56-8f91-47d0-844a-11b7aef1e5ca None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:46.112483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 277/538] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:46.119573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.177407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286d26-ae3b-4fca-8ce9-8f7a4a3ecb4c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.177720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286d26-ae3b-4fca-8ce9-8f7a4a3ecb4c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.178006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286d26-ae3b-4fca-8ce9-8f7a4a3ecb4c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.179408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 278/539] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:46.187530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.187884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.188681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.223194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:46.224432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:46.331913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901110-94b1-4e1d-835c-9d33b6503dc1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.355346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 279/540] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:46.380538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.418537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.419474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.419765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.420087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.437945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:46.439530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:46.468725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:46.469624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:46.470491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:46.474936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 262/541] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:46.479464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.524299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:46.535601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437372a-eb9a-4ff4-be33-c6971a1fa11e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.544422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:46.593773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.594810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.595374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.595861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.605711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5ce43-a3ca-411d-98e1-da5d01fbf586 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.619762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:46.634490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:46.635617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:46.637126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 280/542] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:46.643048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.647969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.648608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.648898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.649181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.650508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:46.650977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:46.652752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:46.666157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 263/543] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:46.675461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.823219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a403c6-4121-4dde-902c-d7195ab84096 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.845639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8922-b4e5-4f96-9741-67dd25a398a6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:46.854979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:46.911305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.911857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.912140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.912419 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.913149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:46.913456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:46.914887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:46.939202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 281/544] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:46.941402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:46.942445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:46.943074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.943354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.943627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.944432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=14ebf672346841c8addaa849fc11a052) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:46.944747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:46.946187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e869cb-0028-4a25-8c98-c3d30cb580e2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:46.953961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 264/545] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/role_assignments?user.id=14ebf672346841c8addaa849fc11a052&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:46.973200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091750-580b-430b-9978-87d4ef1db407 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.973543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091750-580b-430b-9978-87d4ef1db407 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.973783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091750-580b-430b-9978-87d4ef1db407 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.975473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 265/546] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:41:46.981141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:46.981430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:46.981708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:46.982715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.000777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:47.002176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:47.112776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 266/547] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:41:47.142107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.164643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c1d6-c127-4598-9862-08b29a14866f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.240984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.241685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.241974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.242460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.255833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:47.257300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:47.292534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:47.293542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:47.294652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:47.294652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:47.294652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:47.294652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:47.299702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 282/548] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:47.305501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.342492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.352673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee78e3d-53ed-422b-b022-b8b16e58c077 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.413329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.413967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.414316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.414860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.428970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:47.438411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:47.439547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:47.440371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 267/549] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:47.443726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.473014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.539847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0669e92-7e37-4ada-95e6-0e749f1d3754 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.601832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd4b-25ad-459a-8664-06c823d46f23 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.613594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.614238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.614555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.614839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.615660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:47.615978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:47.617514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722ce4a-88f3-4deb-9d83-715e5d935291 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:47.636001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 283/550] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:47.638520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c6b8-fc0e-44f0-81db-63c47ed463c4 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.638520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c6b8-fc0e-44f0-81db-63c47ed463c4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.638520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c6b8-fc0e-44f0-81db-63c47ed463c4 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.639936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 284/551] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:47.640699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c6b8-fc0e-44f0-81db-63c47ed463c4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.650171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.651261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.651781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.652356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.653511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:47.654041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:47.655723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0731d9-b4f7-4146-9c27-306452bbd94e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:47.667071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 268/552] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:47.668833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.669599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.670104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.692070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:47.693642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:47.713806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.762707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c6b8-fc0e-44f0-81db-63c47ed463c4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.776990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 269/553] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:47.779013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.805192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.831196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.831917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.832211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.832536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.847241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:47.849060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:47.861760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:47.862802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:47.863803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 285/554] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:47.864361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:47.904721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3586ef0-526f-48c8-8e75-544f102fad96 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.945421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:47.946467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:47.947069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:47.947922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:47.961068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:47.963602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:47.974203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:47.975636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed970130-6108-4447-bca7-d5969d3e4695 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:47.983940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:47.987039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 270/555] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:47.987896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.015411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.016145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.016377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.016687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.031426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:48.039106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:48.040082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:48.041025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:48.046241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 286/556] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:48.047249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.109015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.116155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9774-1cc7-489b-babd-7cf06763711d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.171272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.172355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.173042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.173525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.174569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:48.175105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.176896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.185117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b0092-08aa-41ab-90aa-1b4edeb03294 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.199320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 271/557] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:41:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:48.200779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.230220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.230934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.231217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.231496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.232214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:48.232525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.233928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.249363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 287/558] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:48.250259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.287814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.335865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade8d52-e750-45e7-b79f-5b4d039ebf3f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.373019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac08d-8b78-4efc-ac6f-2b0860b2c3e0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.381740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.385416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.386508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.386508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.386836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.387570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:48.387772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.389108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.405704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 272/559] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:48.405704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.419556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.421258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.421838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.422311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.514928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.516732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.517275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c8ad6-5aa4-41c1-b90c-f85c12d095b4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.541629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.559040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.559701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.560047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.560335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.571443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.573627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7a6477fa-e061-4ce2-a711-b4e972f7e066 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:48.590407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:48.591434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:48.592350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:48.592877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 273/560] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:48.593796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d89b2-57b4-4b6f-87ac-d35e9001f617 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.594093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d89b2-57b4-4b6f-87ac-d35e9001f617 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.594443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d89b2-57b4-4b6f-87ac-d35e9001f617 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.595867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 274/561] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:48.600328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d89b2-57b4-4b6f-87ac-d35e9001f617 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.642841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 288/562] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:48.644120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.655428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.726541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d89b2-57b4-4b6f-87ac-d35e9001f617 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.762596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d3e3-131f-42f4-be8b-2f87e0c149ca None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:48.770817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.773004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.773491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.774023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.792123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.793565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-2e69f9c0-7f18-428a-9d8f-03a1011a0766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:48.816340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:48.817451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 275/563] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.826445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.830123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:48.830880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:48.831226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:48.831568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:48.832986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:48.833398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:48.834963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:48.848269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:48.850356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:48.865164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 289/564] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:48.865979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.890533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.942449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 276/565] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:48.949384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:48.955401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:48.992339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7b59-0bd4-4c8e-9399-d6bfaa770195 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.034698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.035402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.035776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.036134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.047720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.049188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:49.067652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:49.068620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:49.069532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:49.069532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:49.069532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:49.069532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:49.069532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 290/566] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:49.069814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.093697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0bcd1-45b8-4d23-8540-528349c4a572 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.132866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.133703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.134520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.134930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.135634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:49.135948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.137669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.152726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 277/567] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:49.153579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.182825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.204900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b54d6-0cde-4dd6-b9ba-7019d542b347 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.222649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.250817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.251489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.251769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.252042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.264293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.265830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.280768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e712-4612-4117-a09a-ba01d981b4d1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:49.281739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:49.282658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:49.283550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:49.283550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:49.283550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:49.283550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:49.283550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 291/568] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:49.284264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.323519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.324031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.324318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.324595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.335725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:49.341498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:49.342460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:49.343358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 278/569] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:49.344046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.390454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659c485-9f0c-4929-b30c-03492744bb55 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.425310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.425942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.426243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.426584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.427293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:49.427605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.428988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.429894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.442519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 292/570] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:49.443662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.454386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe922b2b-e031-44a3-a8e9-6d60066058a8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.481801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.499471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.500157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.501735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.501735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.509940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.511429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-36624276-c76e-4ea4-8a3c-c43955729cb2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:49.523133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:49.524160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:49.525291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 279/571] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:49.526111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745704d-320c-4da5-9f1c-8c4e42814874 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.526586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745704d-320c-4da5-9f1c-8c4e42814874 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.526975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745704d-320c-4da5-9f1c-8c4e42814874 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.530381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 280/572] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:49.530381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745704d-320c-4da5-9f1c-8c4e42814874 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.563775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c278-2f41-459b-8d41-c44bf2d7c0a7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.599382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.599918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.600201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.600498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.601169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:49.601536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.604738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.616864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 293/573] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:49.617685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.639505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745704d-320c-4da5-9f1c-8c4e42814874 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.677512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.678134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.678445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.678731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.679456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:49.679768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.681764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.690094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 281/574] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:49.690921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.695935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.714377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.729198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67781f08-4acd-4181-9140-f56c69d08be8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.763729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.764378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.764676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.764956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.771728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.773190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7629ec8c-8743-473a-a8d5-b52e4abe78b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:49.785383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:49.786307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:49.787234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 294/575] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:49.787784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.788328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.788328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.803943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f25fcb-9b3e-4d95-8fe3-859203b130c5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.807107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:49.808313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:49.845314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:49.845976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:49.846262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:49.847260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:49.847593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:49.848044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:49.849652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:49.867788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 282/576] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:49.869647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.882228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 295/577] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:49.883111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:49.939097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.970674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:49.979569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166079-f3cf-4a6c-86e7-d935ecd20e7a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:49.994153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b5b7d-051c-4ebc-ad7f-48459e868faa None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.014597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.015245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.015529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.015806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.022887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.024388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.028413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.029040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.029325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.029600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:50.036032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:50.037111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:50.038003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 283/578] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:50.038955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.045187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.046692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:50.062391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:50.063638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:50.064767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:50.064767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:50.064767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:50.064767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:50.066724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 296/579] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:50.069115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.161661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b531a5b-2ad1-4e98-a215-39ad8912c99c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.184917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.192806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480d5a-9135-4fd4-b1ed-15cf7a3a1653 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.197236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.197862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.198149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.198572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.199524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.199838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.201307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.214751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 284/580] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.215591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.229398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.234955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.235613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.235905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.236183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.250313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.251805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:50.266173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:50.267278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 297/581] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:50.268357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.365138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b46c9-1887-4d03-a952-033b1c99e2e4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.396201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc1e0-2722-4417-ae35-41220b05f0bc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.400357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.401008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.401295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.401574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.412382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:50.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:50.418892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:50.419982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 285/582] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:50.420694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.442493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.443198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.443576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.443917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.444879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.445193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.447671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.461670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 298/583] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.463190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.480504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.508149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.551106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64a127-45b8-478b-84e4-0944fd66f28e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.573631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149c15-9b84-4d00-ac18-89b1b91fc288 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.589223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.589855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.590143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.590453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.591161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.591471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.592918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.604550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 286/584] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.605423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.612819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.613452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.613733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.614013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.614857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.615172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.616626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.622581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 299/585] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/role_assignments?user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.623232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.731544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765353-b6fa-44c6-9ce6-fec0528463af None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.740647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.756635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5a93-727a-470e-b493-7d96a960e928 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.779800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:50.784235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.784957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.785233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.785511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.786292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.786624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.788087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.801694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 287/586] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.802307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.811648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.812308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.812661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.812968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.813707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:50.814045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.815965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:50.830781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 300/587] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:50.831612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:50.935417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1293bd-b516-4ecf-b06c-5b6f6b217592 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.954075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f404e55-eb4a-489f-b25a-2504bbe0a920 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:50.978744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.979485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.979776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.980053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.991524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:50.992206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:50.992481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:50.992760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:50.996835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=1e396c6de22e485bb62fcecfd7f49243, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:50.999276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.001696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.003620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:51.015491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:51.017168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 301/588] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:51.018146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.050156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.079063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 288/589] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:50 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:51.079063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.100036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.153640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5171-6cd9-47a4-a574-57096086dd70 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.199370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.200188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.200538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.200912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.210607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.212296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:51.225800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:51.226815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:51.228338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 302/590] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:51.229085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.236785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1913ed-7c6a-4614-ad28-7bfae4c1477c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.285433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.286528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.286528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.286790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.300319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.300319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:51.319911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:51.320908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:51.321833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:51.321833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:51.321833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:51.321833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:51.326168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 289/591] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:51.327276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.375517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.393502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18194244-af22-4815-b163-cbfdf8fcebb9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.397254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.434326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.435268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.435717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.436241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.437313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:51.437787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.439503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.440938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c763-f068-4be8-9b89-a8c18d9aa66c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.463110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 303/592] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:51.464447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.488434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.489269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.489593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.489937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.490758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:51.491508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.492655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.506575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 290/593] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:51.512588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.579864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aed1-c163-4077-9315-279bfb9c31bf None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.619988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbabb19-065f-4eb8-89f6-a45af63104be None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.620200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.621084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.621621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.622045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.623029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:51.623483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.625020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.638560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 304/594] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:51.639648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.651266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.662403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.663320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.663441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.663754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.675036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.676659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.682746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:51.693477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:51.694603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:51.695556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:51.695556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:51.695556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:51.695556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:51.695556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 291/595] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:51.700258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.763827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14200-36f8-43ae-b714-a72353e0b16b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.811941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.812933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.813289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.813678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/9e8966ba17c04e1782a6ed9fb4e59254/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.827588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e136d-08be-4bca-85e6-3b0fe55f2adb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:51.843659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=9e8966ba17c04e1782a6ed9fb4e59254, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.845219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.870709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:51.870709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:51.871053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:51.871249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:51.880015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:51.881484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:51.904888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:51.906141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 292/596] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:51.911899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.931093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 305/597] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:51 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/9e8966ba17c04e1782a6ed9fb4e59254/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:51.933351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:51.996889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.079969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.112482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191d84-c37e-402a-bd34-e66914134a0d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.156119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30d81-2908-4617-ad9d-bfcb79d71482 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.164446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.165089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.165566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.165855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.166629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:52.166943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.168446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.186394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 293/598] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:52.188185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.215578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.216888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.217380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.217856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.219143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:52.219648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.222639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.250592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 306/599] 10.222.0.85 () {56 vars in 1139 bytes} [Thu Jul 28 17:41:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:52.252668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.394436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.405453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d25b-9508-4205-842c-9201afb1a7dd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.463244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6339f3c-e434-4fa9-ab7a-406ae5ca358a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.477459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.478220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.478556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.478868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.490694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.492161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.496145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:52.519042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:52.520059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:52.520949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:52.521887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 294/600] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:52.528021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.547642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.548308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.548595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.548878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.576496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.578362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:52.605970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:52.607402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:52.608296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:52.608296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:52.608296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:52.608296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:52.613827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 307/601] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:52.619674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.648786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc69e-c9aa-4151-99b6-76d92040d454 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.690287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.690953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.691334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.692079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.693012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:52.693371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.694898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.710529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 295/602] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:52.717073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.751810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.806776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e19ca-b7ff-48ca-b8f2-05de291b7216 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.847973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ce79e-80cc-4160-96dd-af2a3912216f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:52.854500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.855142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.855427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.855711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.856997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:52.857316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.859210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3552b-c5ed-421f-83f0-8638dc4d43b3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:52.879500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 308/603] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:41:52.881164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38c99-9b34-4bc3-b22b-9fc31e9d9801 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.881457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38c99-9b34-4bc3-b22b-9fc31e9d9801 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.882188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38c99-9b34-4bc3-b22b-9fc31e9d9801 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.884570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 309/604] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:52.885316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38c99-9b34-4bc3-b22b-9fc31e9d9801 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:52.900593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:52.901499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:52.901835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:52.902123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/4573990adc20477ab369cd717aaf0679/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:52.918315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:52.930666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=4573990adc20477ab369cd717aaf0679, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:52.932500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b03cf7-8f3b-4ab7-81bd-7d1f5ae1b89c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:53.022763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38c99-9b34-4bc3-b22b-9fc31e9d9801 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.051974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 296/605] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:52 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/4573990adc20477ab369cd717aaf0679/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:53.055014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.055550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.058108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.088302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:53.089092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.089405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.090742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.103015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:53.104519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:53.107687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:53.109057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:53.117519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:53.119208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:53.120344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:53.122530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 310/606] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:53.131490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:53.244996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:53.262253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 297/607] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:53.301617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:53.330928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:53.339078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9395d-536f-4729-a040-a9e6bfcb3e21 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.394670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:53.395316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.395603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.395882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.396839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:53.397154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:53.398827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601eca94-3764-4186-96af-31b71a98e9f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:53.421189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 311/608] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:53.421189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3392-287e-47ba-8479-d3397710bd39 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.421189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3392-287e-47ba-8479-d3397710bd39 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.421189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3392-287e-47ba-8479-d3397710bd39 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.421189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 312/609] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:53.421673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3392-287e-47ba-8479-d3397710bd39 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:53.490464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b00072-ccb9-48bd-b8b1-910272540c15 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.539463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:53.542232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3392-287e-47ba-8479-d3397710bd39 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.542771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.543380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.544268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.560650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0300c3cf-33a4-4922-abd8-daaeb365ba73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:53.568881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:53.570742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:53.572411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:53.575793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 298/610] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:53.577568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.578287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.579071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.584391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:53.585082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.585375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.585673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/14ebf672346841c8addaa849fc11a052/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.603921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=14ebf672346841c8addaa849fc11a052, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:53.605537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:53.608109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:53.609085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:53.611998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:53.682586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:53.698679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 313/611] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:53 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/14ebf672346841c8addaa849fc11a052/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:41:53.698679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:53.721386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 299/612] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:53.748553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:53.934570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce1e94-6f75-4242-ab79-f6177484a6af None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.967726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef5eac-be06-4731-87b8-9744a9fee075 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:53.979739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:53.980370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:53.980656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:53.981298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:53.981662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:53.981970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:53.983438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:53.994778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 314/613] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:53.999702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.011808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.012895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.013390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.013963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.027056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:54.035489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:54.037160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:54.038661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:54.042478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 300/614] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:41:54.048854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.065561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.107674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.113162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2eff-4ac5-4d13-8d5b-e36e964af593 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.151898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.152580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.152864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.153141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.164905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.167322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.178629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f59c7-f0d7-464d-aae1-940393c854d4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:54.183961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:54.184903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:54.185836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:54.185836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:54.185836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:54.185836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:54.189841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 315/615] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:54.193902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.235241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.236346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.236985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.237594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.239092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:54.239602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.241661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.253983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 301/616] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:54.263616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.318681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9acbd-89f7-43ca-a00e-5d03726e33bd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.360960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.361510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.361795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.362075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.362819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:54.363124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.364539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.378751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.382524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 316/617] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:54.384333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.491576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfea66-305a-4266-bd05-93b89a97e3c1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.507190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.543399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.545121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.546090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.546615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.559668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d1c94-39a1-44e4-943f-bbf82e04e2c1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.563543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.565267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7c1f05d1-9378-4a1e-a6a9-0aa56130e993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:54.588985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:54.589943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:54.591086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:54.591086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:54.591086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:54.591086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:54.592670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 302/618] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:54.593824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474394ac-97de-429d-88a6-fbe212d09daa None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.594119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474394ac-97de-429d-88a6-fbe212d09daa None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.594439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474394ac-97de-429d-88a6-fbe212d09daa None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.599942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 303/619] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:54.601046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474394ac-97de-429d-88a6-fbe212d09daa None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.613004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.614984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.615501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.616436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.627659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.631193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b43cd2b8-9095-4c37-bfe2-96390a39d6cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:54.655739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:54.656963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:41:54.658126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:54.665397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 317/620] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:54.667726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.668415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.669276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.690167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:54.691568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:54.722062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.761956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474394ac-97de-429d-88a6-fbe212d09daa None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.783902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 318/621] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:54.785321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.829648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.830480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.830874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.831363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.832066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:54.833381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.834151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:54.848443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 304/622] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:54.850172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:54.880460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:54.929962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6c30-dc84-49bd-81c4-8feccf918864 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:54.980467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:54.981325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:54.981717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:54.982105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:54.983127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:54.983547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:54.985218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.009271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991b1ba-3094-458f-aa4f-0501df54c52f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.015923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 319/623] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:55.016863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.038873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.061548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.063060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.063060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.063369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.078861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:55.084419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:55.085545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:55.086478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:55.093112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 305/624] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:55.094233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.159332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725bd4b-39c3-4116-b21b-a43059538eab None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.206886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.216655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.218149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.218679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.219123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.230655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.231464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be753dc-4700-4900-b71a-d931719cbe8f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.232559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:55.253362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:55.254558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:41:55.255724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:55.259313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 320/625] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:55.261622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.273521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.275193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.275846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.275846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.296134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.299265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:55.331474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:55.332542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:55.333449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:55.333449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:55.333449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:55.333449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:55.342779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 306/626] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:55.344645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.364978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.395164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d753254-61d3-4a32-92b2-4cf14747b3fd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.438968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.439968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.440431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.440981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.442139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:55.442647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.444644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.455754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 321/627] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:55.456905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.521484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec55127-2399-475b-8f3d-cd338de59c92 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.578283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.589502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.590262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.594457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.594457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.594457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:55.594457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.594457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.604149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c301d-e14b-489d-9180-e74ecd20d59e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.617436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 307/628] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:55.619791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.658664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.659386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.659851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.660316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.661381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:55.661893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.664302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498d0a-cdb9-407d-8b60-4c4905f4bad5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.679727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 322/629] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f32e2-55e3-4b51-b03e-14a41cad02f7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f32e2-55e3-4b51-b03e-14a41cad02f7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f32e2-55e3-4b51-b03e-14a41cad02f7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 323/630] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:55.691844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f32e2-55e3-4b51-b03e-14a41cad02f7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.759841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07524-47ee-41b6-96d4-d1f4a6657278 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.798156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.798818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.799151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.799515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.807297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f32e2-55e3-4b51-b03e-14a41cad02f7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:55.812109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.813577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:55.832892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:55.833873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 308/631] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:55.842709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:55.849773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:55.850658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:55.851090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:55.851569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:55.898994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.925840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:55.928641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:55.930163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:55.970460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd3da5-df76-465b-8819-bf0509cf01cf None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.018261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.019201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.019487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.019770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.027085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.028555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a32525e8-1a8f-4c0a-8e09-25f96f404ecc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:56.041361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:56.042297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:56.043215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 309/632] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:56.044519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.044519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.044677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.060461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 324/633] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:56.065028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:56.065115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.065213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:56.146374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 310/634] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:56.146374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.180466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.181069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.185589 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e81c9-3529-4196-8e02-cccee5b0445d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.225511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.226061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.226454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.226667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.227471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:56.228352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.229945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.249733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 325/635] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:56.249733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.273639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e99df6-76f6-4347-b354-f8a48b4cf1ea None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.313480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.314102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.314410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.314710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.315508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:56.315813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.317256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.329237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 311/636] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:56.330068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.360029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb0df-7723-4467-84f9-400faa6ebbdd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.397854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.398743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.399029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.400119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.411976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.413511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:56.429738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:56.430849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 326/637] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:56.432377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.441375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.451299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ac373-0e05-408c-8516-1f5903aa1e2d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.457903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.486504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.486704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.486919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.487190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.497149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-5469a62b-ddc6-41ae-970e-605caecabb3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:56.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:56.503068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:56.503920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 312/638] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:56.504813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453904f9-64d0-4514-805d-21f983a1e5bc None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.504813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453904f9-64d0-4514-805d-21f983a1e5bc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.505336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453904f9-64d0-4514-805d-21f983a1e5bc None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.506399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 313/639] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:56.506896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453904f9-64d0-4514-805d-21f983a1e5bc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.545107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a5fd3-0b24-445c-8345-9ab42c1db9fb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.585721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.586522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.586811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.587659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] PATH_INFO: `/v3/users/glance` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.595599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.597052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:56.608953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:56.610182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 327/640] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:56.611340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.639269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453904f9-64d0-4514-805d-21f983a1e5bc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.674471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.675132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.675416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.676362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/8b5ce51b2ba94ce3ad65f1d61c3efe60/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.688227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.693900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=8b5ce51b2ba94ce3ad65f1d61c3efe60, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.695396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd066dc-df41-4a37-8f8a-a5c9b6733de2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.719775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39371fe-77e4-45c7-898e-45a127c9494f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.754127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.754698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.754984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.755265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.755969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:56.756275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.757653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.777678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 328/641] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:56.778585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.784612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 314/642] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:41:56 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/8b5ce51b2ba94ce3ad65f1d61c3efe60/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:41:56.784612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.784612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.784612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.798821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.814406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:56.815753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:56.910775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 315/643] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:56.913099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:56.937497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc88b8-b7ec-496c-b9b4-dcd462f4df5b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:56.977059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:56.984543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:56.989119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:56.997200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 329/644] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:56.998437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.050728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc54f2-0921-49a3-b83a-11df906b9ace None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.088707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.095790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.096796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.097078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.097661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.098526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:57.100655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.100655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.122318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 316/645] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:57.123741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46801a8-38d3-4282-90be-bda90d41af65 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.127342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.160293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.160919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.161190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.161467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.169590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.171073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:57.183017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:57.184146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:57.185345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 330/646] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:57.189770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.241661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89426b-ad1f-4447-b207-d0368b8b56cd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.252968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.282726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.283387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.283662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.283939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.297876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:57.305298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:57.307068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 317/647] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:57.308082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.315154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d813989-2dd1-4d91-adfd-0ccd46306a6c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.350462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.351106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.351391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.351667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.363190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.364294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.364639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:57.379531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:57.380473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:57.381340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:57.381340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:57.381340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:57.381340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:57.381340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 331/648] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:57.381623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.433711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b866-e087-4de9-b334-b7f6c8b15833 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.472878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.473505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.473793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.474074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.520321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54302bdc-1b2b-45dc-8dfe-ef3a0667487c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.528416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.554772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.558155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.560804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.561466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.561940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.562224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.563905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:57.564288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.565787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.586394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 332/649] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:57.586394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.631445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.702441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 318/650] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:57.703461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.705923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4b357-1958-46f4-bab5-103b6ccc3df5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.749109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.750017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.750467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.750879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.751758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:57.752230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.753925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.769639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 333/651] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:57.770453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.805656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.828974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e4dc-1725-403f-91b1-805fb8f32d65 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.868268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.868926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.869364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.869652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.870380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:57.870700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.872088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.883978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 319/652] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:57.885002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:57.897256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837d7f-b00d-41fc-bee0-613c8472031d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:57.899895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:57.938526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:57.939146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:57.939428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:57.939703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:57.940496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:57.940804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:57.942234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:57.949533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 334/653] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:57.950272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.035649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476db1-0ef4-4feb-a8e0-5b61313aef0d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.072061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.087847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab8a8e-b387-4a03-90c9-f61b62fdb571 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.088094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.088094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.088094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.088094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.098261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.099987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:58.118162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:58.119735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:58.120811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:58.120811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:58.120811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:58.120811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:58.121629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 320/654] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:58.122822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.127512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.128486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.128995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.129450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.142744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.144588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:58.161084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:58.162004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:58.162923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:58.162923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:58.162923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:58.162923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:58.169344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 335/655] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:58.169344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.205510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.263847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0afad8-cb81-42a7-8c17-083df6eb212e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.311911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49022-77ff-4692-98ab-5cc79686762f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.312711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.313730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.314020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.315294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.326156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.328457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:58.345034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:58.346062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:41:58.347023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:58.347777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 321/656] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:58.353419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.361017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.362051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.362569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.363022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.364414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:58.364927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.367390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.385043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 336/657] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:58.386115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.395297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.504317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f60075-9a21-4e42-bc92-ed92c00bad1e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.522935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acce82-8b7d-4862-8f4c-75277fa7df33 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.537377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.554813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.555991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.556319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.556601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.563346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:58.563346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.563346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.578098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 322/658] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:58.579167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.582256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.584201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.584662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.585093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.586076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:58.586576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.588267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.619839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 337/659] 10.222.0.85 () {56 vars in 1137 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:58.621272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.710501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7589402-a921-425d-ae26-a95e36d30ba5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.724738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.753392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.754406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.754908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.755327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.765589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.767751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.781845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21615-ea87-48cc-90c8-c30bed7d8fb3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:41:58.788828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:58.790963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:41:58.791874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:41:58.794540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 323/660] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:58.795705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.848893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.849910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.850374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.850839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:58.860173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:58.861922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:58.867666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:41:58.877616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:58.879030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 338/661] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:58.880442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:58.951828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea934ed-5af5-402e-8a1f-621806be90d5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:58.993324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:58.993832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:58.994119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:58.994438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.018497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7763363-a128-40f5-88d5-8ce1e227a6b4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.046584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.059324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.060255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.060637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.060995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.061894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:59.062340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.064074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053eb18d-6299-4232-a28c-b94e7d28805a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.074678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.076363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.079430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 339/662] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:59.080751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52f87-0e8d-4354-9411-ea119315d01f None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.081456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52f87-0e8d-4354-9411-ea119315d01f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.082025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52f87-0e8d-4354-9411-ea119315d01f None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.086477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 340/663] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:41:59.086477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52f87-0e8d-4354-9411-ea119315d01f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.139356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.236920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52f87-0e8d-4354-9411-ea119315d01f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.243917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 324/664] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:59.243917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.281233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.282015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.282437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.282817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.283916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:59.284534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.287941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f4752-0cc4-463e-a471-f68c3840643e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.317499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 341/665] 10.222.0.85 () {56 vars in 1139 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:59.317499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.317499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.317499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.334280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:41:59.335514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:41:59.376767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7d1c-a97a-4c9c-8bfc-d09de7196592 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.378406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.417814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.418373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.418657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.419184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.419976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=9e8966ba17c04e1782a6ed9fb4e59254) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:41:59.420288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.421706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.424270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 342/666] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:41:59.425575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.438882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 325/667] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/role_assignments?user.id=9e8966ba17c04e1782a6ed9fb4e59254&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:41:59.440017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.455554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.569564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868525fe-13e3-403c-b00e-696ab4992fab None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.572389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce457-ebf8-4a8a-954c-f9335d2e08e6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.617944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.619283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.619580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.619937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.624994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.625750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.626128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.626544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.638583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.640194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:41:59.656344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:59.657512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:59.658752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:41:59.658752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:41:59.658752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:41:59.658752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:59.658752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 343/668] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:59.659356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.691412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.709816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:41:59.711553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:41:59.720572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.782477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f8f6-6b85-493a-a0eb-b370d9dd65d8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.819528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.820276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.820676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.821061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.839309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:41:59.845427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:41:59.846860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:41:59.847738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 344/669] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:41:59.848424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.860823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 326/670] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:41:59.862010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:41:59.946858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.959457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41dfec-c5e8-47b5-b4b1-abafe63953a9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.984133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a677f3-7d36-4b98-b60b-d0fb833f3f75 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:41:59.992447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:41:59.997069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:41:59.998035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:41:59.998506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:41:59.998935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:41:59.999939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.000399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.001925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.015374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 345/671] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.016530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.024981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.025674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.025933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.026214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.026984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.027292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.028777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.043530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 327/672] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.044635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.129680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77032-5c63-4d9f-be44-a3b964fefb3e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.158700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afa8c4-0062-4b1e-bb08-b0906a946781 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.167072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.167999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.168435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.169155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.180648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.182329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.184396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.195226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.195871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.196680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.197009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:00.200462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:00.201522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:00.202608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:00.202608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:00.202608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:00.202608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:00.208541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.208541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.209065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 346/673] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:00.210194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:00.219388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:00.220676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:00.221754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:00.222275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 328/674] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:00.223319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.306799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.326785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373e06f-b339-4ab6-b9a4-9158385039b2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.369472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.370136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.370447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.370733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.382624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.384390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:00.399752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:00.400732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:00.401625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:00.401625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:00.401625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:00.401625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:00.401625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 347/675] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:00.402670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.423218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa415-b383-432a-8443-362cf237adda None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.455906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.459925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.460546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.460832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.461112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.461837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.462292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.467310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ae51f-7732-4e92-8010-bbee069db981 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.475685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 329/676] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.476971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf0c45-1f33-4049-8dc8-2ddc10b7c57a None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.477774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf0c45-1f33-4049-8dc8-2ddc10b7c57a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.478038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf0c45-1f33-4049-8dc8-2ddc10b7c57a None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.482525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 330/677] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:00.482525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf0c45-1f33-4049-8dc8-2ddc10b7c57a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.520999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848e98c-2acc-4ba1-963e-24d606ba70ae None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.557849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.558506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.558797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.559081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.559840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.560156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.561717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.573683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.575409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 348/678] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.576241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.593763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf0c45-1f33-4049-8dc8-2ddc10b7c57a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.631612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.632244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.632533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.632815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.634008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.634389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.635710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec00856-bdc7-45bf-bbe3-b6d60cd66616 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.647302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 331/679] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.651858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.652201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.652492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.671323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:00.672544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:00.689573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f413f-eb18-4486-bdff-0039e6093330 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.698941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.729005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.729582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.729868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.730151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.738753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.740441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.758416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 332/680] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:00.758416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:00.768642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:00.769552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:00.770477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:00.770920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 349/681] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:00.771517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.841063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:00.860747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f13b0e-7be5-49b8-b0f1-f9fab1395748 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.895357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327dfdc-c1e4-42d1-837c-9d693851f5b5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:00.898419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.899063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.899365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.899687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.900486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=4573990adc20477ab369cd717aaf0679) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:00.900804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.902255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.918377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 333/682] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/role_assignments?user.id=4573990adc20477ab369cd717aaf0679&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:00.918377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:00.944898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:00.945663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:00.946626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:00.946987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:00.955807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:00.957448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:00.971471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:00.972704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:00.974078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 350/683] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:00.974917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.015661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.042313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67bb75-4267-4b2d-8c98-8b1506117ace None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.088016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.088663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.088945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.089224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.090640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:01.090952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.092811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.110686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 334/684] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:01.111814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.116959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.125246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d5e5-b47e-47b2-9c0f-d188bc429b9f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.177843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.178911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.178911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.179289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.193766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:01.201894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:01.202951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:01.203815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 351/685] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:01.205010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.246979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e239d6-20e9-46e7-886b-0593ab596022 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.284535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.285106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.285395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.285676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.286518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:01.286836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.288270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.313207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 335/686] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:01.314245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.314918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.337835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176185b6-f00d-49e0-878e-117f03ab8b75 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.385027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.392474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.393722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.394327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.394644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.395614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=14ebf672346841c8addaa849fc11a052) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:01.395938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.398214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.410435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 352/687] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/role_assignments?user.id=14ebf672346841c8addaa849fc11a052&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:01.411398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.454945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a88eb-6a2c-4550-9d1a-3257839f0f77 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.524674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.525415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.525708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.526069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.526860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:01.527174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.528591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.551487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 336/688] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:01.557633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.588219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da9f7c-3c16-4681-bb34-c24ff4435212 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.631916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.632721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.633019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.633305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.658546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.729569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.732023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348ad1d-7f28-4154-9907-192ab041236b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.738306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce12bd6-4df7-43e8-98c3-dc881a360c41 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.766389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.786096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:01.786818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.787110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.787391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.804266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:01.805987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-d0fae8d9-645c-47cd-a4f1-6429c8d15c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:01.825738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:01.826856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:01.827730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:01.827730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:01.827730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:01.827730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:01.836320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 337/689] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:01.838029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0181f-709e-4b73-8566-dd7f67f0cd3f None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.839634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0181f-709e-4b73-8566-dd7f67f0cd3f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.839942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0181f-709e-4b73-8566-dd7f67f0cd3f None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.846184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 338/690] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:01.846184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0181f-709e-4b73-8566-dd7f67f0cd3f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:01.875757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 353/691] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:42:01.877598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:01.878097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:01.878700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:01.898214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:01.901079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:01.953845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:01.968477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0181f-709e-4b73-8566-dd7f67f0cd3f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:01.982287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 354/692] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:01.983538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.045775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.046566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.047231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.049241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.050097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:02.050544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.052024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.070480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 339/693] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:02.070480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.129079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de886be-72ae-4f2c-9ece-313e3c501af5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.150918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.171912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.173348 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.174008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.174451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.188604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.190779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.202316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156f52-d03f-4178-ae97-dd9176fdfdba None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:02.215564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:02.216592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:02.217482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:02.217482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:02.217482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:02.217482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:02.217706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 355/694] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:02.219005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.248846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.249984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.251854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.253226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.264963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.268553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.270226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:02.287851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:02.288787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:02.290460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:02.290460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:02.290460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:02.290460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:02.292684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 340/695] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:02.294176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.347532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a3c9-c351-42fc-8d94-836bc56e804c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.386163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.387285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.387632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.393447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.396379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.397907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.411129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520c68-3cef-4947-a3bf-73b2bb4bce39 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:02.411328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:02.412203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:02.413385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 356/696] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:02.414019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.453490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.454500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.455013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.455492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.456402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:02.456911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.458540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.471173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 341/697] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:02.472343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.515958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.526090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfbf9f-efb6-4fd6-a931-124aec20ccf7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.563822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.564437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.564719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.565005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.565699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:02.566210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.567426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.581597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 357/698] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:02.582468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.583503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342b899-0713-4240-8aec-9e9b103810a0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.621197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.622176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.622707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.623186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.624195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:02.624708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.626439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.645874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 342/699] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:02.647062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.651344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.693557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22bc2-6563-463f-a910-abe0b81b03f5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.732038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.733014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.733014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.733320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.741031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.742538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:02.755382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:02.756314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:02.757256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 358/700] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:02.757745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:02.764609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6789-0e84-499d-98b8-bdb75e30759f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.803298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.804248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:02.805444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.805940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.806461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.807362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:02.809050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:02.877491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6423420-9542-418e-b41e-4d45c4b69d6f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:02.915407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:02.916040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:02.916289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:02.916560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:02.934808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 343/701] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:42:02 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:02.935904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.009839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.011683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a45c69-b670-42bb-b8ed-b5d24dce9b82 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.030733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.205777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.206730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 359/702] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:42:03.209428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e281b14-2e9a-4c09-8e6d-c6d522217df1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.209727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e281b14-2e9a-4c09-8e6d-c6d522217df1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.210378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e281b14-2e9a-4c09-8e6d-c6d522217df1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.216244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 360/703] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:03.219381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e281b14-2e9a-4c09-8e6d-c6d522217df1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.265003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333c17-1104-40b2-a888-1c197e2ef8cf None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.322553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.322553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.322553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.322553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.325077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:03.325077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.325077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a8849-79e1-4e56-9552-e20f98c8fd59 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.349713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 344/704] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:03.351103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.351395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.351675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.371140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:03.372399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:03.373754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e281b14-2e9a-4c09-8e6d-c6d522217df1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.393350 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.419870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.420937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.421544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.422583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.433888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.435553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:03.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:03.455874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:03.456993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:03.456993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:03.456993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:03.456993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:03.463104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 361/705] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:03.463104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.469904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 345/706] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:03.471262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.542034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.625868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03cf6-040d-48f9-8ac3-83b0f30ac87e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.640914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa412-2c96-4eb3-83b8-f2560d221f4b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.677952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.679423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.680243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.681020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.682787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:03.683629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.685902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.694651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 362/707] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:03 2022] GET /identity/v3/role_assignments?user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:03.695938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.701387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.701579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.701782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.701981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.702809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:03.703152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.705291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.723268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 346/708] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:03.733449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:03.760960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.916190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade784e-df6f-47e6-acf3-d3ffe1780983 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.932920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426e23-c143-42dc-8190-8e61a05e90d3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:03.949612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:03.970024 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.970884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.971515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.971809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.972613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:03.974233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:03.984606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:03.985580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.985933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.986278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-69c858a5-8e75-461f-b109-ebdf2177e943 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:03.988823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:03.995221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:03.996269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 363/709] 10.222.0.85 () {56 vars in 1129 bytes} [Thu Jul 28 17:42:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:03.997220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527410-90c3-4c3a-8254-e8b87e11a325 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:03.997220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527410-90c3-4c3a-8254-e8b87e11a325 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:03.997220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527410-90c3-4c3a-8254-e8b87e11a325 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:03.997220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 364/710] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:04.001304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.001304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.006878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.007303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.007721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:04.016902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:04.018079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:04.019088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:04.021061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 347/711] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:04.022119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.039335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:04.041296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:04.133477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 365/712] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:04.134515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.171188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade0597-4fb1-4954-bdcc-47da6dbcc9f3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.177566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:04.217551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.218547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.218838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.219118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.220134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:04.220453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.222381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.230520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 348/713] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:04.231496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.287641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64358b8-d47b-4ed3-850c-277d9c27ade5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.295153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:04.330334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.331001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.331289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.331572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.332342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:04.332656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.334213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.352929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 366/714] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:04.353956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.392944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4421a2-0431-4e4f-943b-d7d35f75f81b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.453327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.454471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.455084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.455686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] PATH_INFO: `/v3/services/network` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.457025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.459040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.480355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:04.482532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:04.483504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:04.484331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:04.490908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 349/715] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:04.492425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.512968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c10f8-4eb1-405d-a94a-8d375f7a4275 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.561888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.563228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.563847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.564312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.565310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:04.565822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.567657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.582605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 367/716] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:04.584676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:04.635732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:04.640608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41dac-3244-40a2-96bb-873ad12121ea None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.689117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.689757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.690043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.690330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.691447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:04.691763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.693430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.710442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 350/717] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:04.792217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c7998-601f-489c-ab46-5789f24b61db None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:04.857037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:04.857686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:04.857976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:04.858384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:04.859710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:04.860079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:04.861517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db812c5-5356-45d6-aa37-6d4ec3d4f8d4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:04.867231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 368/718] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:04.874168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:05.011860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07d6f9-43f9-4e83-9dc5-0948eb70d5c5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:05.049964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:05.050551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:05.050844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:05.051125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:05.051823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] RBAC: Adding query filter params (type=network) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:05.052133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:05.053511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44fc2-1ca1-498f-891b-1d287d639790 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:05.060204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 351/719] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:05.505002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169072-22f8-466d-b384-88c1d90bc41b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:05.505707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169072-22f8-466d-b384-88c1d90bc41b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:05.505707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169072-22f8-466d-b384-88c1d90bc41b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:05.507260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 369/720] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:05.512962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:05.513256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:05.513543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:05.536825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:05.538065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:05.566911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:05.628421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 352/721] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:05.663960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96169072-22f8-466d-b384-88c1d90bc41b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:05.678994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:05.804197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96169072-22f8-466d-b384-88c1d90bc41b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:05.845999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:05.846688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:05.846972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:05.847253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:05.862937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:05.868445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:05.870556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:05.872500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 370/722] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:42:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:05.876140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.000611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36260b-c656-45ee-a5d4-5427bf5a8dd4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.039831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:06.040672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.040833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.041093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.041832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:06.042141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:06.043823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e845-cbdc-41a1-805f-2504e9e1e780 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:06.053069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 353/723] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:06.059241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.161503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:06.258695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f051a9-3409-4dd2-b8a6-04e9ff612fba None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.259037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f051a9-3409-4dd2-b8a6-04e9ff612fba None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.259276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f051a9-3409-4dd2-b8a6-04e9ff612fba None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.261046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 354/724] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:06.266780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.267100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.267328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.301409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:06.302649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:06.328316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:06.343252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790455e-3a5b-43ee-93b2-4653c40bab67 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.407754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:06.408414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.408701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.408980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.424951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 355/725] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:06.430702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:06.432220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:06.454139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:06.456228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:06.458282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:06.458282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:06.458282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:06.458282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:06.458282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 371/726] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:06.463684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.474285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.654623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dba29-61ad-4c48-b500-afcc3043fbca None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.674847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9422-6111-4ad1-9b8e-0e9e254f95bb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.716454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:06.722069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:06.722725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.723012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.723289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.723989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:06.724295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:06.725666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:06.741150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:06.741858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.742150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.742999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.747923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 356/727] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:06.753344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.766136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:06.773426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:06.774318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:06.776562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:06.778536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 372/728] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:06.786373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:06.912782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629bb9b-3442-4df8-8eaf-0fd7ce15c453 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.965684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c366-b3d5-4eea-b59c-c12b932ffc83 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:06.976770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:06.978576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:06.979118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:06.979592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:06.980513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:06.981015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:06.982616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:06.997906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 373/729] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:07.003265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.031994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.033738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.034415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.035985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.049655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.056157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:07.077278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:07.078263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:07.079243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:07.086062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 357/730] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:07.088635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.092642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.193094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5414b4-4d59-448f-b2a3-9d5b9c5562b1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.219817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.259454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.260267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.260548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.260827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.277237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.278719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.284327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761c5de-28af-4fe0-b2d9-a575fce61b85 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:07.309308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:07.310298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:07.311227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:07.311227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:07.311227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:07.311227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:07.311227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 374/731] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:07.316627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.347645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.349276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.350634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.350922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.352366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:07.352688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.354142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef95d-e0eb-4d8a-a675-1992cd8ca0b1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.379756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 358/732] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:07.381082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd739b-b058-48b0-adaf-138725f29191 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.381374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd739b-b058-48b0-adaf-138725f29191 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.381651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd739b-b058-48b0-adaf-138725f29191 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.387048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 359/733] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:07.389241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd739b-b058-48b0-adaf-138725f29191 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.451666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.458965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57101256-5f49-4875-b88b-c363aa3c9301 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.497698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.498726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.499227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.499711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.500711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:07.501224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.502827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354045d-5057-4f90-988a-2ea992202daf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.517502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 375/734] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:07.519096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fafd0-e8b5-4563-bbdf-07ba7106fc06 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.519389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fafd0-e8b5-4563-bbdf-07ba7106fc06 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.519670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fafd0-e8b5-4563-bbdf-07ba7106fc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.531463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd739b-b058-48b0-adaf-138725f29191 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.538209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f47fafd0-e8b5-4563-bbdf-07ba7106fc06 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:07.539637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f47fafd0-e8b5-4563-bbdf-07ba7106fc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:07.564299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.577369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.578269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.578766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.579237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.661265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 376/735] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:07.661265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40669f-99bd-478e-bf0e-6956d4863ecb None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.661265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40669f-99bd-478e-bf0e-6956d4863ecb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.661265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40669f-99bd-478e-bf0e-6956d4863ecb None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.661265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 377/736] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:07.662311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40669f-99bd-478e-bf0e-6956d4863ecb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.682472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.684033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08275a2-4dbd-492c-9372-8454a1c71859 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.758267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.774130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40669f-99bd-478e-bf0e-6956d4863ecb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.783221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 360/737] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:42:07.784486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bd8f3-6803-43d7-bee0-0cff25fa53ab None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.784776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bd8f3-6803-43d7-bee0-0cff25fa53ab None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.785055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bd8f3-6803-43d7-bee0-0cff25fa53ab None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.786467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 361/738] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:07.787327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.787790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.787908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.806906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:07.808098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:07.813171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.813815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.814160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.814474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.821812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.823313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:07.828275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:07.837533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:07.841279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 378/739] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:07.842788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.879952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 362/740] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:07.880896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:07.951021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0f2b-afce-4a5e-8a72-37b1f68a9faa None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:07.985048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:07.985855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:07.985973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:07.986241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:07.986969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:07.988344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.001398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe9a65-91f7-4f02-b93a-a94d03565c44 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.036640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.037217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.037503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.037783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.048674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.050114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-02ff2442-1945-4c3c-8e87-3bce21f6c882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:08.065431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:08.066550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:08.067758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:08.067758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:08.067758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:08.067758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:08.067758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 363/741] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:08.068031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.068110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.068331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.076931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 379/742] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:07 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:08.077856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.088470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:08.089160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.089778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:08.092376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.184419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 364/743] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:42:08.185937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.210041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3252-b8f2-45b0-b59d-b1a411e4327b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.256508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.257139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.257429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.257711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.259188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:08.259504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.260962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.282038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 380/744] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:08.283008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.330141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2b6a3-5b49-4260-9022-b82c6d4dcf52 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.358040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.368269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.368919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.369206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.373069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.373069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.381072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:08.386883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:08.387793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:08.388642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 365/745] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:08.389314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.405310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d448e7-3c48-4cdf-a875-e0233ffb3005 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.443288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.443959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.444232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.444504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.445201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:08.445662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.447431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.464803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 381/746] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:08.464803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.516039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1e6f-ff11-44d9-b771-d53e396cc3cc None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.551107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.551825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.552180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.552532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.553313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.554802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.573323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69804b9d-a850-4edc-8458-2110f9b2f918 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.598250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.609209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.610801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.610801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.610801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.636639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 366/747] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:08 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:08.637718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.711920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.713971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.727294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.814844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a103e-c4b9-49be-9a11-8266aabf3a13 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:08.845035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 382/748] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:42:08.845035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.873175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:08.878773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:08.879509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:08.879854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:08.886065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:08.886065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:08.886065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:08.886065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:08.896586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 367/749] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:08.896586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:08.973841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a96386-8713-4321-8be4-9d8c964a9c4d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.018301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.018971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.019252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.019783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.024293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395123d-38e9-4a58-91da-cf4b03108a81 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.027928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.030375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:09.048156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:09.049217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:09.050227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:09.054245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 383/750] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:09.055667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.056664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.096296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00379777-d184-425a-b07a-49355dec221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.097791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00379777-d184-425a-b07a-49355dec221c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.098811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00379777-d184-425a-b07a-49355dec221c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.099289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00379777-d184-425a-b07a-49355dec221c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.121238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00379777-d184-425a-b07a-49355dec221c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.123268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00379777-d184-425a-b07a-49355dec221c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-00379777-d184-425a-b07a-49355dec221c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:09.140963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:09.141990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:09.142992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:09.142992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:09.142992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:09.142992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:09.144930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 368/751] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:09.145860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00379777-d184-425a-b07a-49355dec221c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.171603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.191828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbd930-c200-4089-984a-ba3e4c9e6ee1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.240467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.241937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.242225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.242815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.255963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.258594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:09.279945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:09.280919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:09.281811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:09.281811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:09.281811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:09.281811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:09.281811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 384/752] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:09.282384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.286399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00379777-d184-425a-b07a-49355dec221c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.333126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.333852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.334213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.335122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.336000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:09.336674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.339515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.358076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 369/753] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:09.359006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.392626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.412889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e0765-6823-4453-b12e-e0ce1624ede5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.452911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.453643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.453940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.454282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.455021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:09.455334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.456832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.459425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.476430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 385/754] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:09.478156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.480969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab75a3-c56b-4e88-b855-ebff346df531 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.519152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.520018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.520453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.520904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.521745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:09.522300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.523948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.541752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 370/755] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:09.543093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.595333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbc99e-5ede-43ca-bce3-355e82c15b41 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.634912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.635539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.635826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.636105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.636757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.638161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=81393bdb74d84cfc886bdf47c2512d8f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:09.638398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.639918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.646224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 386/756] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/role_assignments?user.id=81393bdb74d84cfc886bdf47c2512d8f&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:09.647274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.662157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d8cc4-d783-4c61-b375-ff0dbe53c98b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.718406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:09.736525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.737567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.738006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.738468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.748691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.750480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:09.763823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:09.764789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:09.765694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:09.768247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 371/757] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:09.788041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.792205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09093c01-800c-44fa-af84-ab23522946b1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:09.845115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:09.846946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:09.847234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:09.847514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:09.865165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:09.869509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:09.900109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:09.901113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:09.902833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 387/758] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:09.910178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:09.987871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ee29-5596-4c03-a7d6-ef4f8aedb02d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.025599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b4ce-eb19-4d2d-92a4-01480b4a6cd5 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.026393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.027355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.027888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.028422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.029410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:10.029928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.031704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.050888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.054018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 372/759] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:10.061812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.067924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.071931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.072586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.073046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.073334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.074734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:10.074734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.076185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.097717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 388/760] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:10.103935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.178094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6c1d-20da-48b3-8343-690acdc30ded None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.218184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.219203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.219694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.220194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.220324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5031e0-9457-4015-847f-9383026c6050 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.221320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:10.221939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.223727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436dc90-cc6a-40e7-aa72-aef26bf0e8c1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.230858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 373/761] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/role_assignments?user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:10.287313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.288090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.288409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.288731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.302739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.304403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.311767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5d8a-b3d7-44a9-88f5-680d718f3161 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.312414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5d8a-b3d7-44a9-88f5-680d718f3161 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.312907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5d8a-b3d7-44a9-88f5-680d718f3161 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.314964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 374/762] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:10.327562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:10.328539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:10.330305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:10.330305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:10.330305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:10.330305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:10.330305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 389/763] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:10.332189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.332706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.333193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.338108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.371755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:10.373388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:10.388873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.477854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86d701-f07e-4aca-b862-bb9b9fe527a7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.483209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.503420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 375/764] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:10.521976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.523011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.523598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.524248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.525128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:10.525614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.527253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.540055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.546572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 390/765] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:10.549471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.658824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.667393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca92c-efc4-4c30-b355-80b5acf2ddc2 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.709477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.711015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.711524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.711993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.720762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.722913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.723942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cbe60-02b2-47b6-9b02-fc3465742f23 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:10.738823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:10.739783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:10.740782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:10.744158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 391/766] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:10.751345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.840342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.840731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.841020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.841301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.841942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.844323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-3fa3dd79-8acd-4763-8bf0-94b1307ef8de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:10.858012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:10.859593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:10.861264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 376/767] 10.222.0.85 () {56 vars in 1129 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:10.864524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466d10-cdcd-43a1-9aea-1da96e61ef85 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.864524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466d10-cdcd-43a1-9aea-1da96e61ef85 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.864524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91466d10-cdcd-43a1-9aea-1da96e61ef85 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.864524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 377/768] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:10.867944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466d10-cdcd-43a1-9aea-1da96e61ef85 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:10.925717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4798d-17e1-4ff5-a13e-b9dc42586356 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:10.960057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:10.972213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:10.973200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.974739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.975451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:10.977293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:10.977843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:10.979871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3cf81-48e4-4f05-8426-caffcac9128e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:10.995790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 392/769] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:10.998197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:10.998764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:10.999245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.024555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:11.026445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:11.029543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91466d10-cdcd-43a1-9aea-1da96e61ef85 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.031620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:11.075712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.076996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.077586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.078043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.078966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:11.079455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.082250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.088475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 378/770] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:11.090103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.135730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 393/771] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:11.136786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.211675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353efdf6-ed59-4f42-88f3-a7f95ea36551 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.240707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:11.251617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.252583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.253068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.253562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.256124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=1e396c6de22e485bb62fcecfd7f49243, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.257783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.317010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929781d-eb3e-47cf-a661-d86c601ea07e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.342381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 379/772] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:11 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:11.342381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.380516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.381164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.381950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.382365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.383163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:11.383519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.385094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.391934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 394/773] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:11.403885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.419501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:11.513980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649492-98fc-4cbb-91ef-3fafad86f027 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.561941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.564492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.564986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.565412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] PATH_INFO: `/v3/services/network` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.567111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.569526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:11.586599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:11.587831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:11.588674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:11.594455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 380/774] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:11.596570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896ba78-0668-4d10-bb2b-bc2e569563c1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.598733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.638292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.638950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.639231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.639505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.640277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] RBAC: Adding query filter params (interface=public, service_id=55c8a5c998bd480eb03d8975f006dc39, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:11.640591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.642018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.653874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 395/775] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/endpoints?service_id=55c8a5c998bd480eb03d8975f006dc39&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:11.660546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:11.683948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:11.753205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:11.771603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8440dd6-4725-448b-9dbf-be76d2be94fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.794544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b464d6-8cb4-4506-89dd-16bf989eb500 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:11.834824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.835893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.836857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.837243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.838502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] RBAC: Adding query filter params (name=network) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:11.838944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.840740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.844422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:11.845102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:11.845384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:11.845668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:11.846548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:11.847992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:11.864484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 381/777] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:11.864952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 396/777] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:11.872903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:12.022208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583393-8e78-496a-abb1-425e4d17bbfe None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:12.040742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:12.070766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:12.071755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.072052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.072320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.073021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] RBAC: Adding query filter params (type=network) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:12.073326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:12.074715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:12.088426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 382/778] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:12.095076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:12.225373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc6aa-c81d-4702-a0f8-69dc2e8e19ec None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:12.260171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:12.265310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:12.265935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.266219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.266524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.267288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] RBAC: Adding query filter params (interface=public, service_id=8267a52944344d05ba0a7e386f498d9a, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:12.267601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:12.269043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8fb95-9103-4d71-8987-17a6340e3e61 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:12.276974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 397/779] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:12 2022] GET /identity/v3/endpoints?service_id=8267a52944344d05ba0a7e386f498d9a&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:12.283504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:12.376041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22f9c-c173-40fe-a6f6-14251ddada87 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.376041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22f9c-c173-40fe-a6f6-14251ddada87 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.376041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e22f9c-c173-40fe-a6f6-14251ddada87 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.377083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 398/780] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:12.382303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0574f-9cfc-4d03-a0db-47a4d55e0268 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.382601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0574f-9cfc-4d03-a0db-47a4d55e0268 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.383409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0574f-9cfc-4d03-a0db-47a4d55e0268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.416701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f2f0574f-9cfc-4d03-a0db-47a4d55e0268 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:12.417922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f2f0574f-9cfc-4d03-a0db-47a4d55e0268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:12.424852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19451bc-194a-4613-ae46-a56ed7968852 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:12.493283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:12.493910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.494194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.494497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.495158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:12.496529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:12.503876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 383/781] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:12 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:12.552471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 399/782] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:12.578254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:12.644981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:12.742089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ebd9-0f7a-4d29-89c2-5209da09b200 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:12.802981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:12.803578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.803959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.804151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.807779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d37d12-dd4e-401c-ab05-f620b7379ba3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.808079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d37d12-dd4e-401c-ab05-f620b7379ba3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.808363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d37d12-dd4e-401c-ab05-f620b7379ba3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.810594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 400/783] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:12.817335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:12.817747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:12.818624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:12.822326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:12.827984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:12.829874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:12.831674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 384/784] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:12.848402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:12.849052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:12.850328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:12.990313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 401/785] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:13.013390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.068058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f6838-bd2b-4d8f-9e48-15f4d8ad059c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.070924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.132637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.143764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.144403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.144692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.144974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.145706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:13.146016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:13.147438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:13.157368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 385/786] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:42:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:13.167440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.204450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24172869-6441-426e-bcfe-cdb8c490a4fe None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.265287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.265959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.266257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.266636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.287733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:13.297708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:13.299989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:13.301922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:13.310373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 402/787] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:13.310373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.324657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f11e7-8a62-4d72-92e1-c6ea00107d63 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.404733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.405400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.405686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.405964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.436063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:13.437895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:13.464575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:13.465544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:13.466477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:13.466477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:13.466477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:13.466477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:13.473953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 386/788] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:13.478102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.497334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33395cd-9f7b-4fa6-a505-301de58f4f8b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.522555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.537540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.574507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.575578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.576074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.576637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.577580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:13.578096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:13.579771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:13.605866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 403/789] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:13.608721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.697487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8c678-93fa-478e-8497-57e24e0e09b7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.771655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.772283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.772565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.772859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.774380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:13.774694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:13.777368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:13.804395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 387/790] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:13.812272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da714379-77eb-49e8-af0c-21458020b0c1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.814854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.868002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.869411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.869924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.870429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.885282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:13.886830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-f8ea418e-abe5-4116-ae15-6b948aeab34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:13.902900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:13.903829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:13.904704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:13.904704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:13.904704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:13.904704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:13.904704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 404/791] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:13.905718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4fa7-abd9-4702-aad5-ad4c1e131c1b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.906008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4fa7-abd9-4702-aad5-ad4c1e131c1b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.906284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4fa7-abd9-4702-aad5-ad4c1e131c1b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:13.911791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 405/792] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:13.912556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4fa7-abd9-4702-aad5-ad4c1e131c1b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:13.918809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.950139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad1e88-334f-4a0b-b837-aa697942543b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:13.952440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:13.995732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:13.997267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:13.997734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:13.998201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.009599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.011638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e44722c9-a2ba-4b17-9a2d-0658194f0b44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:14.027557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:14.028539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:14.029527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:14.037226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 388/793] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:14.037226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.037226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.037226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.057490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:14.058442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4fa7-abd9-4702-aad5-ad4c1e131c1b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.058727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:14.097496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.098474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.098959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.099443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.100354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:14.100855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.103135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.116635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 406/794] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:14.119161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.136687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 389/795] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:14.143162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.223770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.235687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3134-6087-46a4-af31-b7fd51108f97 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.243883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.276704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.277692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.278174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.278691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.279760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:14.280278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.280931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d4d3-4040-4e96-a42a-f88732b09281 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.282217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.306578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 407/796] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:14.307770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.321103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.321483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.321706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.321987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.331876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.333335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:14.349279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:14.350209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:14.351196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 390/797] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:14.352639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.432744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137cdbc-e5cb-47c4-918b-779eb159eab9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.471856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.473059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.473545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.474034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.481704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30af8-8b8e-41c5-8081-6c5a5e19e396 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.486994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:14.492228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:14.493784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:14.494651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:14.497363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 408/798] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:14.498509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.511987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.512392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.524221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.524876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.525162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.525442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.543692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.546049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:14.567246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:14.568206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:14.569065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:14.569065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:14.569065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:14.569065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:14.571558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 391/799] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:14.571745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.622832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8723f74-0a78-4646-a994-110890547c0d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.658418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.659040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.659323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.659601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.660398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:14.660707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.662137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed665d8-9587-4a51-85fb-65e40f44dae8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.683596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 409/800] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:14.685219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292b3d-492d-4042-bda0-b9a0bf716f45 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.685722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292b3d-492d-4042-bda0-b9a0bf716f45 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.686204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292b3d-492d-4042-bda0-b9a0bf716f45 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.688070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 410/801] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:14.689024 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292b3d-492d-4042-bda0-b9a0bf716f45 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.701669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053074b-8ec4-400e-80e4-14a7024603d6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.739400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.740025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.740416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.740701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.741416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:14.741723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.743553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.756887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 392/802] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:14.756887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:14.770399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.812876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:14.814424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292b3d-492d-4042-bda0-b9a0bf716f45 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.869826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.870894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.871182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.872343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.872343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:14.872693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.874135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d54b86-181b-4125-b61b-68c45df824ab None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.897843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 411/803] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:14.897843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.897843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.897843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.913591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f3036-8ff6-4b7f-a11a-9dfc6324eabd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:14.916459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:14.917678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:14.951428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:14.952085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:14.952381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:14.952661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:14.963380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:14.964821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:14.979416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:14.981388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 393/804] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:14.983254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.002609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 412/805] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:15.003558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.065656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.083322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.092279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f614bc1-5040-43b8-ad63-5bd6c3767adb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.111667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00abf9-afcd-48d0-846a-9227c85d86c2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.134866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.135844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.136326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.136829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.149178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.150669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.154823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.155464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.155743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.156014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.162603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.164725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-efdcf85e-c1f1-482a-86ef-2f6f9fd9e05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:15.165813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.167570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 394/806] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277787-b348-45e0-a934-5ac2f908873d None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277787-b348-45e0-a934-5ac2f908873d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.169777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc277787-b348-45e0-a934-5ac2f908873d None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.170962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 395/807] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:15.171089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc277787-b348-45e0-a934-5ac2f908873d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:15.184332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.185433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:15.186922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 413/808] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.187414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.281601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc277787-b348-45e0-a934-5ac2f908873d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.292078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eb76d-072c-443e-a6b4-892f805b836b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.309447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.316331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.316956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.317239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.317518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.318211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:15.318601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.319994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.327911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.328545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.328831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.329103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.329732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.331113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.332387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 396/809] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:15.333211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-64863ace-adad-40f6-8ed9-d5e0ff42d703 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.342100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:15.344110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 414/810] 10.222.0.85 () {56 vars in 1129 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.345858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.346137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.346439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.349097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.363747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:15.364935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:15.432163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 415/811] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:15.433016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.459864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d532160-2945-4c77-afbd-2f8dd9e059af None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.494534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.495261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.495555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.495828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.496517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:15.496824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.498214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.509864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 397/812] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:15.510734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.537710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80cd7e-ef8a-41b1-8b30-7db790156cb9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.549106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.574428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.575054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.575348 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.575619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.575733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.576373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:15.576684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.578160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.592005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 416/813] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:15.593161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.626214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fa6cd-8bf4-4ffd-b653-c0d29b89e3b8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.673315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.674232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.674738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.675248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.682406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.683980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:15.695535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.696595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:15.697470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 398/814] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.697950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.704817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585875f9-8ba7-4cc7-8f88-17ae40890f97 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.740706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.741355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.741643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.741925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.742878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:15.743186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.744572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.749042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 417/815] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:15.749868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.795396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.821711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7001-7b3a-454f-b3dd-fe5a2b14cd66 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.848309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:15.860323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.860976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.861258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.861539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.862174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.863597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.865257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86dc52-bbe4-41f7-aff5-ff90a7fb8906 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.872622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.873644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:15.874508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 399/816] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.875393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.901807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:15.902482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:15.902769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:15.903047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:15.910505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:15.911963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:15.924714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:15.925876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 418/817] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:15.926977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:15.988687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85908f-2806-4914-9c87-ac0a42f384b1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.024584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.025492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.025785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.026061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/81393bdb74d84cfc886bdf47c2512d8f/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.032892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.040250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9add3c-a24a-4fcd-9f08-fbf2d9c5bead None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.041891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=81393bdb74d84cfc886bdf47c2512d8f, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.043789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.080770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.081709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.082078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.082746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.083546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:16.083865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.085339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.098066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 419/818] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:16.098923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.135168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 400/819] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:15 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/81393bdb74d84cfc886bdf47c2512d8f/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:16.136040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.192902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.218047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ac4ed-42cb-4409-8c46-f1f0390be0e0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.271434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.271985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.272294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.272543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.273856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:16.274170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.275694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.275923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c71b6-bf7e-43ed-8eee-386bdfd4eee3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.286487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 420/820] 10.222.0.85 () {56 vars in 1124 bytes} [Thu Jul 28 17:42:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:16.286487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.312053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.325272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.325894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.326179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.326492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.327282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] RBAC: Adding query filter params (name=network) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:16.327588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.328970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.334931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 401/821] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:16.336047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.406108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e99e-2307-441e-bf1f-6f2b6c29aa08 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.452653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.453460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.453573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.453845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.454701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:16.454994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.456418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.457622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737012-6df6-4147-8df0-c00fdde8a3bf None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.460171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.476055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 421/822] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:16.482682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.504862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.505666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.505951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.506225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/8b5ce51b2ba94ce3ad65f1d61c3efe60/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.508214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=8b5ce51b2ba94ce3ad65f1d61c3efe60, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.509791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.560330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.582394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 402/823] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:16 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/8b5ce51b2ba94ce3ad65f1d61c3efe60/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:16.582394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.599289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d19af-a1a6-431a-9a5c-bdb3a67a653e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.642036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.643050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.643427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.643780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.644716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.646142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.721689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c90fd-9eb0-4d75-bd78-816529abbaad None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.761592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 422/824] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:16.762840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.774543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.776026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.776523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.777003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.777935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:16.778613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.780292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:16.787134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 403/825] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:16.794447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:16.839606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.916258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda08c27-f29c-498f-805b-c6e81573b850 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.944950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6af004-fcd5-4f3e-8e44-04c261feb4f5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:16.984502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:16.987579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:16.988592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:16.989115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:16.989588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:16.990555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:16.992300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:17.000307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:17.000913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:17.001204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:17.001486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:17.089577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:17.091305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:17.106262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 404/826] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:17.228297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 423/827] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:42:17.234569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:17.370040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:17.403264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450083-45e9-43b7-9c57-2a90e03658d9 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:17.441802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:17.442555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:17.442995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:17.443472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:17.457503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:17.459203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:17.475041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:17.476879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:17.478452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:17.478452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:17.478452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:17.478452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:17.478452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 405/828] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:17.482788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:17.530706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:17.609583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f0c23-f6c2-487f-8286-6030723f16ae None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:17.654591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:17.655134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:17.655420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:17.655702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:17.656554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:17.656858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:17.658372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:17.672922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 424/829] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:17.678530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:17.785941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c05b61-e458-4d36-95bb-00f25bc366b6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:17.818755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:17.829760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:17.830501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:17.838012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:17.838012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:17.845441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:17.846943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:17.859867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:17.860918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:17.862104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 406/830] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:17.866835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:17.973465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:17.984009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf320-4d0e-4291-a1a4-1e836b5ae3db None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:18.025974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:18.026645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.026932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.027257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.028104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:18.028418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:18.029874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7541a8-b547-4ecb-a5aa-936ec4210fc9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:18.044397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 425/831] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:18.050084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:18.159923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e54ae0-9396-4a38-879c-4106702a9eac None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:18.196556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:18.197250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.198253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.198253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.198613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:18.198931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:18.200373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849fa2-65cc-4220-9f1e-3d0ce067df58 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:18.207263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 407/832] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:18 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:18.289987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a446-29cb-42a2-b347-576d022bdaac None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.290411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a446-29cb-42a2-b347-576d022bdaac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.290600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a446-29cb-42a2-b347-576d022bdaac None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.292416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 426/833] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:18.298392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.298702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.298969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.319534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:18.321340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:18.385791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:18.409675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 408/834] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:18.433949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515a446-29cb-42a2-b347-576d022bdaac None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:18.547005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515a446-29cb-42a2-b347-576d022bdaac None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:18.588392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:18.589377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.589849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.590302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.604789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:18.610092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:18.611062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:42:18.611900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:18.614205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 427/835] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:18.622914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:18.727374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4905ec-ecca-43fd-9324-3fced1f88778 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:18.764199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:18.764824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.765111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.765392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.766123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:18.766461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:18.767851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e045fbe-f799-46dd-ab03-f66cdf2d757d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:18.777566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 409/836] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:18.782991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:18.800128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:18.900408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4df5a-45bd-45d8-830a-98bcb1f85801 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:18.934328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4df3a-ecd1-48b3-a84b-ecc38b66c019 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.934777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4df3a-ecd1-48b3-a84b-ecc38b66c019 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.935149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4df3a-ecd1-48b3-a84b-ecc38b66c019 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.938527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 410/837] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:18.939173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:18.939819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.940105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.940605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.941744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:18.942030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:18.942310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:18.952966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:18.954468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:18.962017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:18.963384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:18.970854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:18.971774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:18.972731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:18.972731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:18.972731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:18.972731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:18.972731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 428/838] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:18.978794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.007951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.045286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 411/839] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:19.058566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.075318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.108421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f52a36-6eb9-42ba-8eb0-35b8e2f7d69e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.151546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.152356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.152642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.152916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.153624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:19.153929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.155329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.173464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 429/840] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:19.174518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.206148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095e281-2385-4ef8-aa95-502689cd8f43 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.245430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.246086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.246399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.246684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.259367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:19.264467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:19.266395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:19.267516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 412/841] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:19.272233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.310790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657f326-4e7a-4d74-8f42-797b5fc60abe None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.316574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.340229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.356819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.357582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.357873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.358155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.366770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.369219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-461f63b0-af11-46ad-89bd-879561f702ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:19.383920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:19.385273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 430/842] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91f9f-9885-4304-b26b-798f646ee38e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.386935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91f9f-9885-4304-b26b-798f646ee38e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.387630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91f9f-9885-4304-b26b-798f646ee38e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.388611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 431/843] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:19.391521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde91f9f-9885-4304-b26b-798f646ee38e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.409245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f8622-92c5-4eda-bc5b-138434d6753a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.454596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.455473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.455776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.456101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.456731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:19.457037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.458417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8348c-82dc-410f-897f-1e4e2b952a37 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.473988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 413/844] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:19.473988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.473988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.473988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.494589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:19.495844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:19.527612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde91f9f-9885-4304-b26b-798f646ee38e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.569934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.571610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.571908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.572163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.572978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:19.573292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.574782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.580410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 414/845] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:19.581583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.594638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 432/846] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:19.600433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.622581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.656704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.717696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4a29-71c5-4085-b84e-4dae858017d1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.737262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a3ece-62bb-4d7c-94c2-deff89ee6df0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.758446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.759108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.759393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.759668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.772990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.775239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.783196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.783824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.784110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.784380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:19.792135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:19.793309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:19.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:19.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:19.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:19.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:19.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 415/847] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:19.795154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:19.872302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.873938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.910587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.926745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:19.935015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8096892-8bb4-4e4c-a9f1-03cc6c19a3ec None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:19.974584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:19.975269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:19.975558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:19.975843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:19.976513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:19.978015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:19.988142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:19.990065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:19.991110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 416/848] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:19.992129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.006519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 433/849] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:20.008594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.114939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799c541-fc11-4027-af73-57b24db8765f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.130464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19536-b5c9-4f9b-b245-19dbf6b303e9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.153182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.153822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.154107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.154410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.155119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:20.155430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.156810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.166950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.167465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.168351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.168658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.168927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.169601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:20.169911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.171334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.176534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 417/850] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:20.176534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.182678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 434/851] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:20.183377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.193095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.307194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa37dd-c3cf-4919-a983-bbc46ebff9fe None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.307956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab093030-e07b-4fee-a1a9-36e0e6729d88 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.346971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.347733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.348124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.348472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.350777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.352239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.352529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.352808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] PATH_INFO: `/v3/users/swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.360609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.361970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.362111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.364264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:20.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:20.376038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:42:20.377001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:20.381156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 435/852] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:20.381630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:20.391451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:20.394033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:20.394033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:20.394033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:20.394033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:20.394723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.394943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 418/853] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:20.394943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.465021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.467052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.503308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0720-543c-4542-a186-36b2e0fa500e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.521192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dc74e-5382-4c33-8456-0f78d9875c61 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.542622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.543505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.543804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.544089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.544777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:20.545088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.546542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.551108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 436/854] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:20.551986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.567082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.567726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.568017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.568304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.569314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:20.570681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.572660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d61fe-4af2-4cb6-a1b1-69ce3a706048 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.604870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 419/855] 10.222.0.85 () {56 vars in 1135 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:20.604870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5aee7-b50c-40f3-a8ea-53b362e3a390 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.604870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5aee7-b50c-40f3-a8ea-53b362e3a390 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.604870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5aee7-b50c-40f3-a8ea-53b362e3a390 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.604870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 420/856] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:20.611312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5aee7-b50c-40f3-a8ea-53b362e3a390 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.674595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9860d12-85a4-4510-87de-b94fdd8a9ed3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.710459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.720248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.720879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.724452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.724452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.724452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:20.724452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.724452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94b88f-23d1-4d11-8670-2f1e0552fbe8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.742886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5aee7-b50c-40f3-a8ea-53b362e3a390 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.744158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 437/857] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:20.746394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.746394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.747205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.758476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:20.769967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:20.771217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:20.783771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:20.784402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:20.784788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:20.785076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:20.866535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 438/858] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:20.868174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:20.881875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:20.884332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:20.987759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c08de-f596-455f-bbef-2d20b652f26a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:20.997292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 421/859] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:42:20.998277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.011810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.013391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.029987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.031020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.031770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.031878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.041809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.044165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:21.056467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:21.057436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 439/860] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:21.059320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.120952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e29b-1dce-486e-9deb-7de88800eeea None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.163793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.165013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.165533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.165873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.166560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.168428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:21.177423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:21.178649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:21.179750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 422/861] 10.222.0.85 () {56 vars in 1132 bytes} [Thu Jul 28 17:42:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:21.181016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.192769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea6cd9-68b4-4786-a24c-7918cadb4d6e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.234065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.234822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.235089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.235361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.248520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.249909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:21.266735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:21.267712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 440/862] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:21.274033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.281752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.308219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810f911-4605-422b-8a51-3d9c7be9edf5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.316068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.358999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.359853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.360237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.360640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.361792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:21.362240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.364404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.380022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 423/863] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:21.381225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.403267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392451-075c-4b2a-9466-63f2afd6cf91 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.441726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.442377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.442935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.442935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.443843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:21.444046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.445595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.450756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 441/864] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:21.451405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.507411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4f7d9-a87f-4b61-86b5-198d4920636d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.549614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.550308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.550621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.550905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.551666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:21.552001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.553394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.555750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.563895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:21.567939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 424/865] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:21.568962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.575746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18c43-b6d4-4f3a-acc8-2b7a4fa0ed98 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.617833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.618649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.619054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.619426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.620393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=81393bdb74d84cfc886bdf47c2512d8f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:21.620801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.622342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.629136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 442/866] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/role_assignments?user.id=81393bdb74d84cfc886bdf47c2512d8f&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:21.635406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.708773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a898fd-79f2-4f58-a6d3-4f8e16cb8bd0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.759003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.759930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.760562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.761021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.761915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:21.762436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.764114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.775013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 425/867] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:21.793875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbff6-1463-4b59-8a31-9b1154911cb1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:21.848642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:21.849330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:21.849642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:21.849926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:21.861903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:21.863492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:21.877366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:21.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:21.879302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 443/868] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:21.885571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:21.968729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:22.020580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa984-6d7b-4c29-9708-3221b3492db7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:22.058448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:22.059081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:22.059368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:22.059649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:22.060415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:22.060727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:22.062164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b324119-d6ab-4ebc-ab42-c1602e718278 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:22.076634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 426/869] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:22.082864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:22.111277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:22.201389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dafa5d-4315-4fe2-9dee-266dc3b5af67 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:22.238093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:22.239132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:22.239642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:22.240248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:22.241340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:22.241979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:22.243761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63499813-73ed-4db3-a6b5-0b28a69f34ac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:22.258578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 444/870] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:22 2022] GET /identity/v3/role_assignments?user.id=8b5ce51b2ba94ce3ad65f1d61c3efe60&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:22.734100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e46a-526c-4b59-9ea0-c3c6997321cb None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:22.734673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e46a-526c-4b59-9ea0-c3c6997321cb None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:22.734673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e46a-526c-4b59-9ea0-c3c6997321cb None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:22.736323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 427/871] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:22.741871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c648a7-23a6-4a67-a18f-1c3e22936ed9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:22.742193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c648a7-23a6-4a67-a18f-1c3e22936ed9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:22.742517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c648a7-23a6-4a67-a18f-1c3e22936ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:22.768077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c0c648a7-23a6-4a67-a18f-1c3e22936ed9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:22.770047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c0c648a7-23a6-4a67-a18f-1c3e22936ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:22.869528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 445/872] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:22.898814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181e46a-526c-4b59-9ea0-c3c6997321cb None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:23.034844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181e46a-526c-4b59-9ea0-c3c6997321cb None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:23.056860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:23.077109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:23.078137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:23.078672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:23.079170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:23.080095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:23.081818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f20c06-4ad9-4647-88d7-8ed46d2fc4cb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:23.157438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 428/873] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:22 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:23.996028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bcf7-0605-4141-81c9-bf34b1c5bb9d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:23.997053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bcf7-0605-4141-81c9-bf34b1c5bb9d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:23.997546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bcf7-0605-4141-81c9-bf34b1c5bb9d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:23.999399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 446/874] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:24.005298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.006217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.006854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.023843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fe18-10f4-436d-99d4-ae096c81d9e1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.024179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fe18-10f4-436d-99d4-ae096c81d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.024410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fe18-10f4-436d-99d4-ae096c81d9e1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.026080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 447/875] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:24.028713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:24.030174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:24.031568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.031860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.032145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.052402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:24.053681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:24.064696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.120659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 429/876] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:24.137280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 448/877] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:24.145851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.160813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.265721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ea02-dd6d-4583-825e-253234f207b4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.268730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.275136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f4309-a423-45a6-b1dd-e36cce68cf7c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.308886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.309903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.310301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.310724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.313415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.314293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.314875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.315402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.316451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:24.318068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cc4b8-9e8d-419f-81b4-0f96aea6deaf None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:24.325394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:24.330482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:24.331480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:24.332439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:24.338813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 430/878] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:24.339032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.446537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 449/879] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:24 2022] POST /identity/v3/services => generated 269 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:24.453000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ff05-09a6-4afa-bb93-2541a71efd0c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.453282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ff05-09a6-4afa-bb93-2541a71efd0c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.453569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ff05-09a6-4afa-bb93-2541a71efd0c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.459578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 450/880] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:24.461492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.461777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.462064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.470851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.474876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13224c-4bec-4317-adb6-ec03e6c36aff None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.486951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:24.488202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:24.540528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.541508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.541976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.543280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.544319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:24.544823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:24.546691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:24.559373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 431/881] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:24.567548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.592469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 451/882] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:24.605120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.611876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.705063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104cfa-a93b-4092-a454-d6d620a94c20 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.740092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df37ca9-dc06-4980-8583-5b1fd3001cab None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.750203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.750999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.751294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.751578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.763662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:24.765205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:24.781411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.782135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.782482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.782769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:24.783000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:24.784107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:24.784993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:24.784993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:24.784993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:24.784993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:24.785208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:24.785208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:24.785345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 432/883] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:24.792127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.794877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:24.801168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:24.802211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:24.803090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:24.805715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 452/884] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:24.812381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.899659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:24.940399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e45b4-be2c-4398-8bd2-6635fadc122c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.948655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc894f-6aa6-4353-bdd0-78a9a8c15e04 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.996139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.997029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.997486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.998036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.999231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:25.000124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.001957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.008646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.009820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.010765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.010967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.012222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:25.012860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.015478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.017385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 433/885] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:25.025611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.032900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 453/886] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:25.041282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.142648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.172030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ff958-2753-480f-b425-01f19a20b4af None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.211146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff2d-7a09-49f9-a1f1-ea8f334b73f8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.218669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.219965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.220742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.221380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.224535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.232784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.234665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:25.248731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:25.249775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:25.250768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:25.255458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 434/887] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:25.259113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.260671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.261319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.261604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.261881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.262609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:25.262919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.265644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d615712-f974-4343-b7cd-f17cf2c80d35 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.287471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 454/888] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:25.290938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6c8-3643-47da-9ce7-78060f722e74 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.291379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6c8-3643-47da-9ce7-78060f722e74 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.291823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ee6c8-3643-47da-9ce7-78060f722e74 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.300288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 455/889] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:25.302567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.302983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.303160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.328688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:25.330280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:25.404063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e88f77-1e56-4e6a-a3a6-7e72c218e552 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.426886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 456/890] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:25.447166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.453616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.454620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.455112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.455573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.456564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:25.457066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.458752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.489656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 435/891] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:25.495077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.525604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.570073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.576479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07acd1-f026-4e34-a90f-bfee3b6e18bd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.623950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.624898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.625478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.625914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.627286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.628826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.629277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179c29-07d3-4899-8ed3-2719937d41c8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:25.643353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:25.644608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:25.645622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:25.653728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 457/892] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:25.654822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.685337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.687111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.687839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.688826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.779232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.780717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2f92-3fa2-4c5f-9f50-edf61438c9bc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:25.780978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.806731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.843908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:25.844553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:25.844842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:25.845123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:25.845818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:25.846180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:25.847990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:25.854129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 458/893] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:25.855793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:25.859341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:25.923295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 436/894] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:25.924475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.003915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5257ed-6573-4263-8475-c2a38089f019 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.050903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.052314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.052424 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f288c-ac1a-4b9f-a077-0c1b680a7aec None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.053194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.054460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.056301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:26.056847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.058476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.066025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 459/895] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:26.072277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.105297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.107111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.107606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.108103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.122963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.125168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.127338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:26.148616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:26.149737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:26.150768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:26.150768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:26.150768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:26.150768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:26.154681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 437/896] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:26.158100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.167093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.189942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc966ad-f14f-47ec-9429-8bd08fade389 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.228382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.229280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.229280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.229579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.230338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] RBAC: Adding query filter params (interface=public, service_id=c546dafcd7454481aa6e237cc5636811, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:26.230675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.232049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.243179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 460/897] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/endpoints?service_id=c546dafcd7454481aa6e237cc5636811&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:26.244084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.281740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d0fc-6e9a-4a7a-9e27-b7f73c149675 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.318770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.319816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.320306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.320797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.321794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:26.322304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.323924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.342013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 438/898] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:26.343261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.359105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cdc92-4ec0-4a83-9a62-0bbd3e7ff151 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.373704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.397489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.399055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.399480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.399760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.400481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.402133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.409835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 461/899] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:42:26.412039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.459537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15a693-c237-4f0a-99bc-f913975e4663 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.500157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.501097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.501536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.501969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.511347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.513140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-d3e75ab4-6807-4fc0-8dae-2ec3c78f2a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:26.527045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:26.528182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:26.529285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:26.531467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 439/900] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:26.537503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.654646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3620-feec-493f-b3fe-b07b02c41620 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.654904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3620-feec-493f-b3fe-b07b02c41620 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.655144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a3620-feec-493f-b3fe-b07b02c41620 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.656354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d48498-6501-4cfe-9c69-51fabdf261ed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.656778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 440/901] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:26.661777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.662313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.662838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.684705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:26.686316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:26.693025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.693654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.693941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.694224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.695582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:26.695896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.697521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.710892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 462/902] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:26.717849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.761302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.783329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 441/903] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:26.807026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:26.859664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5e15-6475-47e9-9d4a-eb4e4009d749 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.874205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.903690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.905567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.906067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.906685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.907801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] RBAC: Adding query filter params (role.id=9b534570538f4486bca5138b598faa6b, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=81393bdb74d84cfc886bdf47c2512d8f) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:26.908398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.910283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:26.918030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 463/904] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/role_assignments?user.id=81393bdb74d84cfc886bdf47c2512d8f&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=9b534570538f4486bca5138b598faa6b => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:26.944680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b7bf9-cce0-489c-906d-77f78bb20c13 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:26.987314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:26.987990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.988273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.988553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.989208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:26.990656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:27.001497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:27.002692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:27.003748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 442/905] 10.222.0.85 () {56 vars in 1132 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:27.011162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:27.126960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:27.130793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772f761-ac64-4f91-b9d4-a0de94042d3f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:27.178093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:27.178675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.179019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.179411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.180553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:27.180933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:27.182330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.188151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 464/906] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:27.193984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:27.320221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1437-2bba-49b1-a069-97bf4b1742c0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:27.341668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:27.361567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:27.362527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.362957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.363379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.364211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:27.364718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:27.366399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2278f-7244-479a-aaed-66ef7721ed77 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.373467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 443/907] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:27.381810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:27.425245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe3947-bc69-42b5-a3be-161044c7900d None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.425604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe3947-bc69-42b5-a3be-161044c7900d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.425818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe3947-bc69-42b5-a3be-161044c7900d None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.427487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 444/908] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:27.432968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.433199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.433482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.455763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:27.457278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:27.497899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81e1e7-7e81-42b8-8bc5-9e71a65a8fff None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:27.542432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 445/909] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:27.548409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:27.549043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.549329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.549609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.550471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] RBAC: Adding query filter params (interface=public, service_id=89d5cb6ebc444e6bb6549204226ccb6b, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:27.550793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:27.552217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.558319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 465/910] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:27 2022] GET /identity/v3/endpoints?service_id=89d5cb6ebc444e6bb6549204226ccb6b&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:27.563562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:27.567897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:27.607332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:27.692842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea4ee-b77d-4941-8cc6-ef77b17a730f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:27.700156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67362d-5847-41ce-8324-78b9df9edc3f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:27.717427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:27.735852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:27.736650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.737131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.737427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.738447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:27.739906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bc5f5-4097-4eff-8517-1a1a53a829dc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.747505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:27.748332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:27.749012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:27.749553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:27.750572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:27.752512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d59333-0777-412e-8459-ab08b3ff68d2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:27.758843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 446/911] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:27 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:42:27.802745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 466/912] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:27 2022] POST /identity/v3/services => generated 241 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:28.543563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a748-3e78-44ac-ab55-1697e45ca9d5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:28.544088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a748-3e78-44ac-ab55-1697e45ca9d5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:28.544194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a748-3e78-44ac-ab55-1697e45ca9d5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:28.545711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 447/913] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:28.550982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:28.551827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:28.552309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:28.571442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:28.573165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:28.651612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 467/914] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:28.675261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466a748-3e78-44ac-ab55-1697e45ca9d5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:28.794923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466a748-3e78-44ac-ab55-1697e45ca9d5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:28.831676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:28.832702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:28.833188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:28.833666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:28.834557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:28.836163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:28.846044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:28.847042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:28.847918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 448/915] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:28.853964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:28.869197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:28.966752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bb87-74c9-41e6-b1d4-96cd14b1f10b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.005420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.006457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.007551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.008176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.009129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:29.009644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.011307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f7b17-6494-4d92-b1bd-00b0976dfa40 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.017027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 468/916] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:29.024827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.026058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b31c5-fdef-473e-98c5-cfe3a7654e8e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.026369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b31c5-fdef-473e-98c5-cfe3a7654e8e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.026663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b31c5-fdef-473e-98c5-cfe3a7654e8e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.028226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 469/917] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:29.033036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.033464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.033746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.056052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:29.057495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:29.072211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.141039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5011cc-1489-433b-8935-10ec479ccf96 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.148224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 470/918] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:29.169981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.176943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.177565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.177850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.178131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.179427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:29.179942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.181324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.183835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.188042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 449/919] 10.222.0.85 () {56 vars in 1114 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:29.193460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.295865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f113934-1c7b-4f33-8853-a8045aa93874 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.303432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b6aa1-e67e-4c0d-a645-ab7cd77cbfde None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.307691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.344170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.345113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.345453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.345733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.346019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.346301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.346573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.347038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.347139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.348423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.361235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:29.366819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:29.368025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:29.368847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:29.369470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 471/920] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:29.375475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.431770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 450/921] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:42:29.468777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.508975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95473145-2951-4bb9-b421-10446e16f6fd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.555363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.556044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.556044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.556358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.558387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:29.558647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.560116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.572579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 472/922] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:29.579021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.693893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9fc83-2e63-4bba-a7b6-e0341b295287 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.735465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.736125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.736411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.736692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.749074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.750602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-2bb7eff0-7269-4e4b-b4c0-b08acfc6d706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:29.766707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:29.767686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:29.768585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:29.768585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:29.768585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:29.768585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:29.768585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 451/923] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:29.775767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:29.880145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c511a-bb94-4d93-8da7-a9b5f83b117c None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.880480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c511a-bb94-4d93-8da7-a9b5f83b117c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.880744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c511a-bb94-4d93-8da7-a9b5f83b117c None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.886464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 452/924] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:29.888080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688efba-2eae-4262-871e-932c9bb8e7cc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.888370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688efba-2eae-4262-871e-932c9bb8e7cc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.888652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688efba-2eae-4262-871e-932c9bb8e7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.889711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10165-e123-49cc-9216-6cb089adc4d6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:29.907766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d688efba-2eae-4262-871e-932c9bb8e7cc None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:29.908965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d688efba-2eae-4262-871e-932c9bb8e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:29.917906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.919356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:29.927778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:29.933485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:29.945526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 473/925] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:29.951548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.002669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 453/926] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:30.003977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76253e2-5f8c-4c2c-99df-8e7d26010913 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.004276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76253e2-5f8c-4c2c-99df-8e7d26010913 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.004558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76253e2-5f8c-4c2c-99df-8e7d26010913 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.005899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 454/927] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:30.012200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.012768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.013667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.035897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:30.037102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:30.066805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd08f2c-bd52-4a21-9d93-8f140190954a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.103976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.104549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.104930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.105118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.113142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.118612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 455/928] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:30.118751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.118852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:30.134287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:30.135484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:30.136766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 474/929] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:30.139384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.191187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.202365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.234901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a28e14-d854-4a3d-bd19-d0694cf69e9b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.250736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556ad2-1259-44cb-a649-c48ba4c28c78 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.274653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.275308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.275592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.275877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.276510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.277879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.286559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:30.286897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:30.287805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:42:30.295110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:30.296252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.296252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.296252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.296574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 456/930] 10.222.0.85 () {56 vars in 1132 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:30.296694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.296801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.296893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:30.307008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:30.308028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:30.308911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:30.313062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 475/931] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:30.313904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.408677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f543b69-59aa-4f37-afc9-b126526af791 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.418149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84df4f-c165-430f-82ad-60df4cdb65b4 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.431068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.447619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.448579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.448851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.449130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.449930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.450241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.451707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.454281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.457314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.458102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.458675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.458858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.459525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.459836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.461331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.466198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 476/932] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.467025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.473898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 457/933] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.474863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.575322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42064d-27fc-48fc-97c4-cfecefe90b28 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.590284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791de2f-b20c-443c-9636-06bc85d012cb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.614501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.615215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.615594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.615950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.617475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.617796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.619219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.624707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 477/934] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.627486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.629026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.629026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.629226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.629300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.629369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.629631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.631031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.638620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 458/935] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.643809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.676918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.708833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.743015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a273e1-42b6-47bd-90ad-8b59fb0ee940 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.755091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cff46-762c-4043-aaca-ad27425487f3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.780500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.781075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.781361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.781647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.792473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.793801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.794187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.794454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.794743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.795023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.795708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.796026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.797371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.802430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 459/936] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.803171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:30.814806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:30.815711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:30.816604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:30.816604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:30.816604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:30.816604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:30.817237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 478/937] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:30.817968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:30.919728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.919935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb462444-c543-4d28-8ee9-9931962749c6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.933081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b45232-d643-4a33-bca5-e77a35abb240 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:30.947127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:30.958857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.961810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.961810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.961810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.961810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.962241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.966229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:30.966856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:30.967138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:30.967584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:30.968365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] RBAC: Adding query filter params (interface=public, service_id=80bf0718ef6c4b3faff68cc24618c3fd, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:30.968682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:30.970036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:30.975514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 479/938] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/endpoints?service_id=80bf0718ef6c4b3faff68cc24618c3fd&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:30.981252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.039506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 460/939] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:31.040340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.116156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d960b18-bd75-4e87-954b-07f622af5301 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.165834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.166502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.166786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.167402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.168208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:31.168516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.169968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.180520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33619-815b-40d7-8292-f0c0eb656222 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.185370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 480/940] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:31.191651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.205152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:31.231349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.231976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.232259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.232539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.233378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.235022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.243789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 461/941] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:31 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:42:31.318748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09417-98e5-4a8b-8eb8-b796aa212ba1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.358779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.359447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.359771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.360049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.368689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.370177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:31.384018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:31.384984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:31.386080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 481/942] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:31.391368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.440888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:31.503278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bde61b-5dcc-4b7b-b37f-8dba824b0e38 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.539499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.540127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.540416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.540696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.541454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:31.541824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.543595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb07c3d-175f-425a-a537-95fde2598235 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.557187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 462/943] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:31.563170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.608188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7368-2b2e-47fa-a544-185ce55a8603 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.608701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7368-2b2e-47fa-a544-185ce55a8603 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.608832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7368-2b2e-47fa-a544-185ce55a8603 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.610204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 463/944] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:31.615513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.616045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.617312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.646021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:31.647251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:31.653656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:31.693151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c6364-042a-4d7d-a29f-ffd8b4f04922 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.738320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 464/945] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:31.738549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.739356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.739631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.739975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/81393bdb74d84cfc886bdf47c2512d8f/roles/9b534570538f4486bca5138b598faa6b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.742382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=81393bdb74d84cfc886bdf47c2512d8f, role_id=9b534570538f4486bca5138b598faa6b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.743909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.762912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:31.800562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 482/946] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:31 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/81393bdb74d84cfc886bdf47c2512d8f/roles/9b534570538f4486bca5138b598faa6b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:31.808526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:31.902263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b473b5-ef5f-4c3e-a3f9-5600205ebcca None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:31.941694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:31.942338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:31.942660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:31.942943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:31.943775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:31.946017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:31.948781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING keystone.server.flask.common [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 17:42:32.002186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.sql.core [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:42:32.002186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:42:32.002186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [parameters: {'id': 'ddf8b6e92edf4447bf19092a0c9b66fd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:42:32.002186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92084) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddf8b6e92edf4447bf19092a0c9b66fd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddf8b6e92edf4447bf19092a0c9b66fd. Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:42:32.010712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 17:42:32.015819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application yield resource Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise self.value Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application session.commit() Jul 28 17:42:32.016761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:42:32.017672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:42:32.019234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:42:32.020035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [parameters: {'id': 'ddf8b6e92edf4447bf19092a0c9b66fd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:32.020895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:32.021746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ddf8b6e92edf4447bf19092a0c9b66fd. Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:32.024080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 465/947] 10.222.0.85 () {60 vars in 1140 bytes} [Thu Jul 28 17:42:31 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:32.025368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:32.125908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f49158-ce1b-40c9-b94c-ed5a20eae3e0 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:32.156334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:32.162372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:32.163120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:32.163869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:32.163869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:32.171105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:32.172555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-bf0bdaa9-f1f4-45ec-899d-d81b4f4a6f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:32.184779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:32.186847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:32.189061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 483/948] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:32.197122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:32.289941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:32.304412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f9a-4ca4-4329-8fd7-9bfaac38d5db None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:32.342021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:32.342679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:32.342962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:32.347068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:32.347068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:32.347068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:32.347068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb172-2515-40fd-adc7-79e9786ea549 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:32.367402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 466/949] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:33.217928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94bb83-2060-4768-9e98-d70f601872de None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.218560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94bb83-2060-4768-9e98-d70f601872de None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.218560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94bb83-2060-4768-9e98-d70f601872de None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.220170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 484/950] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:33.225960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f6aff-4d76-436f-b435-dca32f341eee None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.226722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f6aff-4d76-436f-b435-dca32f341eee None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.226722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f6aff-4d76-436f-b435-dca32f341eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.247058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e08f6aff-4d76-436f-b435-dca32f341eee None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:33.248575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e08f6aff-4d76-436f-b435-dca32f341eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:33.326469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 467/951] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:33.344952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94bb83-2060-4768-9e98-d70f601872de None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:33.422896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c5cb-201b-4f25-bcf2-5bf02013e048 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.423226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c5cb-201b-4f25-bcf2-5bf02013e048 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.423452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c5cb-201b-4f25-bcf2-5bf02013e048 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.424919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 468/952] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:33.435918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.436189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.438903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.454788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94bb83-2060-4768-9e98-d70f601872de None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:33.461394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:33.462767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:33.498021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:33.498703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.498993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.499542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.500206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:33.502308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:33.514859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:33.516259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:33.517371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 485/953] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:33.523178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:33.562382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 469/954] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:33.581230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:33.595046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:33.606723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:33.668030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc40248-4291-4df6-9f5c-ec11aa24ef1d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:33.695638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bd831-5f99-4c6c-b98e-b719b7aa16cc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:33.715527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:33.716632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.716997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.717354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.718121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:33.718538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:33.720551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:33.727819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 486/955] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:33.733809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:33.735513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:33.736347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.736793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.737219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.738009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:33.739762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:33.750281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:33.751589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:33.752638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:33.761130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 470/956] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:33.762299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:33.851164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ef8b-5c0e-491f-98be-f0a2c3108f49 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:33.855653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:33.870786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:33.891408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:33.892096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.892658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.892896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.893593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] RBAC: Adding query filter params (type=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:33.893974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:33.895322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaeb83e-3b73-457e-82d9-d0b62694ec8f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:33.900447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b60a-9143-48ce-b7ec-be64cca35e2e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:33.904610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 487/957] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:33.921265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c898b-a1c5-457f-9676-1d0191978b48 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.921603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c898b-a1c5-457f-9676-1d0191978b48 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.921860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c898b-a1c5-457f-9676-1d0191978b48 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.923629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 488/958] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:33.930141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.931196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.931714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.943958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:33.944931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:33.945291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:33.945626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:33.946439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:33.946833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:33.948303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:33.954551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 471/959] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:33.957394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:33.958649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:33.960079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.054987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 489/960] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:34.073478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.091695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddeba4-0516-480d-87f1-7cc3ed473a84 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.137466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.138473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.138942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.139626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.140850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:34.141309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.142855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.146099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.152366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 472/961] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:34.158473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.212211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567bbfe-b838-4bc1-bb7c-f0b18c0b9fdc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.231027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.260211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.261276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.261764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.262230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.273848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:34.279341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:34.280272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:34.281091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:34.289335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077efac-fa4e-4056-b5db-a2831e8fe3b7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.290434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 490/962] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:34.292047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.334594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.335752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.336056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.336366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.337107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.339237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.404124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68535fd-c85e-4dbf-b2f5-5fc276508994 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.441289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.441912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.442193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.442501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.443221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:34.449827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.449827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847e82-33d8-4a46-a8ab-00696643003a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.451091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 473/963] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:34.462910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 491/964] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:34.464388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.519068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8d1d-d33a-452a-b47b-7671dcac8974 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.519386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8d1d-d33a-452a-b47b-7671dcac8974 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.519624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c8d1d-d33a-452a-b47b-7671dcac8974 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.521233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 492/965] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:34.526892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.527219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.527447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.532645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.552167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:34.553406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:34.589431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3ed81-6461-4023-98d2-ce64f4539458 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.595065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.628233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.628896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.629201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.629508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.644086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.645892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.659612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 493/966] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:34.664936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:34.665835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:34.666775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:34.666775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:34.666775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:34.666775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.668830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 474/967] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:34.674182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.686675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.785873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff7ffc-49cd-44b2-bbd3-c26d31a371f4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.817765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da184dc-aef6-4a21-85bc-5b5dbf2fbb0e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:34.821189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.826855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.827681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.828059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.828435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.829234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:34.829637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.831261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.847573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 494/968] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:34.853090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.869905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:34.870943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:34.871488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:34.872094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:34.873113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:34.875119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:34.882240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:34.921161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.sql.core [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:42:34.921161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:42:34.921161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [parameters: {'user_id': '8b1b102ae82a4b4594f7663d55bc3e27', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:42:34.921161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92084) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:42:34.932097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:42:34.933077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application yield resource Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise self.value Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application session.commit() Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:42:34.934046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:42:34.934968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:42:34.935870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application result.read() Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:42:34.936740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [parameters: {'user_id': '8b1b102ae82a4b4594f7663d55bc3e27', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:34.937591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 17:42:34.940454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:34.942745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 475/969] 10.222.0.85 () {60 vars in 1131 bytes} [Thu Jul 28 17:42:34 2022] POST /identity/v3/users => generated 160 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:34.946134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:34.974239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeab1-7e8b-453f-95e2-e3cf66087578 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.012068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.013067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.013579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.014157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.023676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.025481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:35.038933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:35.039907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:42:35.040846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:35.048282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 495/970] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:35.049535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:35.061921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a911-dbc2-47dc-b786-024a8887106c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.104057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.104729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.105018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.105295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.113465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.115422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:35.133043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:35.134045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:42:35.135019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:35.136065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 476/971] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:35.140377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:35.151427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:35.170015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e1b1-691c-401d-96b3-0adc7cdc514b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.181143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:35.209177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.210479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.211171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.211649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.212646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:35.213161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.214824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.236318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 496/972] 10.222.0.85 () {56 vars in 1149 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:35.245798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:35.271909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32bce6-95a8-4fd3-9844-543416d53ee3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.318799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.320348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.320829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.321355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.322379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:35.322812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.325335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.373972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bad76-7530-45bb-9740-b7a9635bab73 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.383253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 477/973] 10.222.0.85 () {56 vars in 1101 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:35.423313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.424246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.424686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.425108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.441013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:35.509329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.511119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.628119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 497/974] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:42:35.635553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:35.736943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:35.753795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684644-dcc8-4d19-9192-65cf6a5a0ad0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.794037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.794715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.795005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.795283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.805914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.807388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:35.822556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:35.823535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:35.824408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:35.824408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:35.824408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:35.824408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:35.824408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 478/975] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:35.829253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:35.878707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:35.938170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3c7e5-3896-4052-9802-a42c3461c41e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:35.974298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:35.975226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:35.975662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:35.975943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:35.976647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:35.976961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:35.978565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe1669-75ed-498c-bf66-55a1580bc3f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:35.995246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 498/976] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:35.996498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.014900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768576-3186-40ca-962e-7f0897188d1c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.015195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768576-3186-40ca-962e-7f0897188d1c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.015476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab768576-3186-40ca-962e-7f0897188d1c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.017137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 499/977] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:36.021814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.022104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.022697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.041820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:36.043424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:36.102400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe472af8-e6de-4c4e-b0c0-2b8dd9d09a02 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.122649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 500/978] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:36.138663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.139478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.139769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.140051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.140298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.144092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:36.148597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.150643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:36.163307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:36.164222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:36.166153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 479/979] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:36.170526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.182006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:36.259905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a2237-6797-4408-a2ad-a7dcd6988398 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.285761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621d925-812c-46f8-b0b3-822567f04802 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.299094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.299731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.300017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.300296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.300997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.302407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b47964-ca9f-49c2-96af-1f5b47ab982c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.324628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.325261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.325550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.325857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.326878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:36.327196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.328626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.341923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 480/980] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:36.347885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.394427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 501/981] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:36 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:42:36.410889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:36.463643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe53e-9bf6-4a52-8c52-af38c0208a09 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.511166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.511796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.512075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.512348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.513155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] RBAC: Adding query filter params (role.id=9b534570538f4486bca5138b598faa6b, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=81393bdb74d84cfc886bdf47c2512d8f) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:36.513465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.515010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc919af-2c67-49ee-a5d1-d7093592fd8e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.530389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 481/982] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/role_assignments?user.id=81393bdb74d84cfc886bdf47c2512d8f&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=9b534570538f4486bca5138b598faa6b => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:36.631335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6461e-cd46-49bc-bed9-c51f79e815b0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.631685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6461e-cd46-49bc-bed9-c51f79e815b0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.632038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6461e-cd46-49bc-bed9-c51f79e815b0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.633595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 502/983] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:36.639237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.639534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.639807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.660591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:36.662084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:36.738145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 482/984] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:36.760417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6461e-cd46-49bc-bed9-c51f79e815b0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.867914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:36.877922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6461e-cd46-49bc-bed9-c51f79e815b0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.914631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.915206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.915551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.915828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.916409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.918057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:36.927210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:36.928189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:36.929431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 503/985] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:36.934807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.985261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:37.039602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546184f-b774-4ebe-95a1-2ad2a8175a6b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:37.074792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:37.075429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.075718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.075994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.076683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:37.076989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:37.078377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:37.089090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 483/986] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:37.090250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:37.193575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab889-77cc-4d5e-a9b3-03d7a41eccb8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:37.229370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:37.230312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.230835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.231249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.232076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:37.232536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:37.234107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:37.242740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 504/987] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:37.246027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:37.357777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31d36-6b96-403d-91ef-06464123587c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:37.369219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:37.393326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:37.394272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.394764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.395188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.396192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] RBAC: Adding query filter params (interface=admin, service_id=80bf0718ef6c4b3faff68cc24618c3fd, region_id=RegionOne) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:37.396652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:37.398270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68a3ae-1d30-4f1b-9107-9a98134b00c4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:37.407867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 484/988] 10.222.0.85 () {56 vars in 1249 bytes} [Thu Jul 28 17:42:37 2022] GET /identity/v3/endpoints?service_id=80bf0718ef6c4b3faff68cc24618c3fd&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:37.413205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:37.437654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:37.520689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e055a4-b7ee-4745-b751-4bec83aaaa07 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:37.557736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:37.558378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.558663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.558942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.559660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:37.561092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64fcc0-f5e7-4122-a245-e30ecd59d20f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:37.572943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 505/989] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:37 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:42:38.452327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985292-c931-4cd2-b9ce-5fb1819e76da None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.452856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985292-c931-4cd2-b9ce-5fb1819e76da None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.452962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985292-c931-4cd2-b9ce-5fb1819e76da None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.454409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 485/990] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:38.459457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3c6a-def2-4de0-93c9-d0b6687682fe None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.459914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3c6a-def2-4de0-93c9-d0b6687682fe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.460166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3c6a-def2-4de0-93c9-d0b6687682fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.478746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f89b3c6a-def2-4de0-93c9-d0b6687682fe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:38.479951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f89b3c6a-def2-4de0-93c9-d0b6687682fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:38.550927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 506/991] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:38.574561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985292-c931-4cd2-b9ce-5fb1819e76da None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:38.581753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117181c-2831-4bff-920d-9c20be4b2a1d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.582274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117181c-2831-4bff-920d-9c20be4b2a1d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.582876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117181c-2831-4bff-920d-9c20be4b2a1d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.584422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 507/992] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:38.590168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.590497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.590782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.610793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:38.611994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:38.623676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:38.685809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985292-c931-4cd2-b9ce-5fb1819e76da None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:38.686724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 508/993] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:38.708990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:38.723702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:38.724617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.724617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.724898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.726030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:38.727603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:38.737072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:38.738093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:38.739794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 486/994] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:38.744784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:38.790429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:38.819543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292fb7d-b76b-4708-80ae-c17cc44f14ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:38.858240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:38.858943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.859237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.859525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] PATH_INFO: `/v3/services/image` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.860156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:38.861603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:38.862826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ffcd7-7077-4ab9-b41b-b062bfe84613 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:38.871270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:38.872266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:38.873210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 509/995] 10.222.0.85 () {56 vars in 1105 bytes} [Thu Jul 28 17:42:38 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:38.878380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:38.891664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:38.903072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:38.903906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:38.904187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:38.904461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:38.905137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:38.905438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:38.906842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:38.911492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 487/996] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:38.917140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:38.995924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd9022-c637-4b30-be29-9b553bb2d078 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.031260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf5fc7-4982-4ced-ab8a-ef6ab0e655a3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.033358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:39.035269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.036119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.036278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.036422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.037096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] RBAC: Adding query filter params (name=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:39.037411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.038856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.044593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 510/997] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:39.050881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:39.082958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.083771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.084596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.084994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.086044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:39.086521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.088446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.099984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 488/998] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:39.101219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:39.134534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:39.165795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720627-8489-4f47-abd9-12e8fc66abb3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.200717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.201346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.201630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.201906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.202626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] RBAC: Adding query filter params (type=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:39.202938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.204315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.209583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 511/999] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:39.214073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef970-f58f-42f1-be45-4b75a7c58023 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.255387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.256273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.257270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.257735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.258721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] RBAC: Adding query filter params (interface=public, service_id=d5e6f91f74264dd49cf66738ca94b347, region_id=RegionOne) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:39.259199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.260789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb7fa3-87f5-4229-8121-2a1117623b7c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.275079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 489/1000] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:39 2022] GET /identity/v3/endpoints?service_id=d5e6f91f74264dd49cf66738ca94b347&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:39.280397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:39.391176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab2a7-0b6d-4468-8d7c-ce1dd3e015c1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.429179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.429803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.430090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.430388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.431050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.432423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2b36-ba67-4887-abf4-842f74a8897c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.434595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:39.446383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 512/1001] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:39 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:42:39.610221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f579ec-4c11-46f2-a08a-1d9d5220ae59 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.611303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f579ec-4c11-46f2-a08a-1d9d5220ae59 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.611797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f579ec-4c11-46f2-a08a-1d9d5220ae59 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.613619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 490/1002] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:39.619812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.620317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.620407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.640936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:39.642143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:39.712928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 513/1003] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:39.735067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f579ec-4c11-46f2-a08a-1d9d5220ae59 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:39.767586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:39.852263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f579ec-4c11-46f2-a08a-1d9d5220ae59 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:39.888444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:39.888998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:39.889281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:39.889558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:39.890187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:39.891582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:39.901516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:39.902548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:39.903562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 491/1004] 10.222.0.85 () {56 vars in 1126 bytes} [Thu Jul 28 17:42:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:39.910852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:40.016849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b198d11-3440-4cf5-ba2e-6ed7f6d38529 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:40.047295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:40.052918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:40.053846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:40.054181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:40.054486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:40.055170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:40.055476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:40.056834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:40.065156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 514/1005] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:40.067541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:40.169544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:40.176311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d5780-c238-4555-91a9-cc7ff4901bbd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:40.211826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:40.212958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:40.213824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:40.214650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:40.215336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:40.215652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:40.217017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f424d15-2fb1-4d94-901d-b336bf6f9a09 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:40.226400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 492/1006] 10.222.0.85 () {56 vars in 1122 bytes} [Thu Jul 28 17:42:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:40.228548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:40.334923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ecb23-66b3-4b38-97f2-912c689e2946 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:40.373071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:40.373852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:40.374135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:40.374526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:40.375333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:40.376837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea530d3-b641-434e-9e80-96b94e74c002 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:40.494652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 515/1007] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:41.268900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57bee-f716-4732-87a5-feec57aeba0e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.269416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57bee-f716-4732-87a5-feec57aeba0e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.269493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57bee-f716-4732-87a5-feec57aeba0e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.270989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 493/1008] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:41.276071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c886-d435-45ad-8748-dad6ec221432 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.276470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c886-d435-45ad-8748-dad6ec221432 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.276643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c886-d435-45ad-8748-dad6ec221432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.295362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-62a2c886-d435-45ad-8748-dad6ec221432 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:41.296572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62a2c886-d435-45ad-8748-dad6ec221432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:41.349833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:41.375080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 516/1009] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:41.398313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57bee-f716-4732-87a5-feec57aeba0e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:41.472562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15fbd-394a-4a40-984f-5e0d6817eabf None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.472906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15fbd-394a-4a40-984f-5e0d6817eabf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.473135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c15fbd-394a-4a40-984f-5e0d6817eabf None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.474527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 517/1010] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:41.479558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.479841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.480123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.497871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:41.499098 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:41.509768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57bee-f716-4732-87a5-feec57aeba0e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:41.544955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:41.545511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.545762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.546042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.546785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:41.548153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:41.571315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 518/1011] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:41.594297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:41.656437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 494/1012] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:41 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:41.714793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34fd5a-d8d1-4b00-883d-1b0cb46a5d68 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:41.728295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:41.762923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:41.763664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.764375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.764375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] PATH_INFO: `/v3/services/compute` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.765206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:41.766738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:41.777656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:41.784102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:41.785162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 519/1013] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:41.787562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:41.813368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:41.903543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abe90-ed7f-4892-9756-ffbf4d9a417f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:41.937936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:41.938598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.938883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.939164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.939851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] RBAC: Adding query filter params (name=compute) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:41.940158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:41.941558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:41.946769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 495/1014] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:41.952432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:42.055752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fb7dd-aba9-4c1e-9043-3053055d7aed None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:42.090168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:42.090742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.091091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.091378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.092064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:42.092372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:42.093758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8ad38-3b5c-483d-9f8d-ee666e164621 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:42.098930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 520/1015] 10.222.0.85 () {56 vars in 1112 bytes} [Thu Jul 28 17:42:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:42.105363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:42.192409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:42.212398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8a366-dc28-4972-acda-f55e6d25d79c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:42.255314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:42.256871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.257427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.257936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.258901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:42.260543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaa613-85f6-4fbe-8a2c-d0eb2508769f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:42.357266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 496/1016] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:42:42.543266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686a08-240e-47da-a37c-1387916a85d7 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.543764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686a08-240e-47da-a37c-1387916a85d7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.543866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686a08-240e-47da-a37c-1387916a85d7 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.545538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 521/1017] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:42.551422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.552066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.552066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.577434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:42.578676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:42.660692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 497/1018] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:42.684978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95686a08-240e-47da-a37c-1387916a85d7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:42.723485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:42.791757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95686a08-240e-47da-a37c-1387916a85d7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:42.833619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:42.834328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.837681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.837681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.838759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:42.840484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751316-64e8-4ebc-9ae8-dd2659a20f22 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:42.842048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 522/1019] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:42.847907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:42.911087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:42.952352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501946e-51a1-4fe3-b826-5e1aafa3dc7a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:42.987849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:42.988476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:42.988761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:42.989038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:42.989735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:42.991874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0992-64ea-4443-ac04-c1a7ebdf7edc None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:43.123921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 498/1020] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:42:42 2022] POST /identity/v3/projects => generated 311 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:42:43.748090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9270-2631-4022-b581-c640cb7900b1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:43.748636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9270-2631-4022-b581-c640cb7900b1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:43.748636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f9270-2631-4022-b581-c640cb7900b1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:43.750136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 523/1021] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:43.755218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:43.755589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:43.755800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:43.774633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:43.775847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:43.854397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 499/1022] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:43.874598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f9270-2631-4022-b581-c640cb7900b1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:43.982595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f9270-2631-4022-b581-c640cb7900b1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:43.984119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.017966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.018610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.018893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.019165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] PATH_INFO: `/v3/services/image` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.019789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:44.021139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:44.030188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:44.031176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:44.036933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 524/1023] 10.222.0.85 () {56 vars in 1105 bytes} [Thu Jul 28 17:42:43 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:44.040355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.089930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.142837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cf64-db7f-4e53-9e29-4f8795ab8d7f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:44.178916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.179477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.179760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.180033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.180708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] RBAC: Adding query filter params (name=image) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:44.181011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:44.182381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:44.191890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 500/1024] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:44.193102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.296176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd138d83-f776-43cc-9960-59222f33b473 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:44.333476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.334090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.334495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.334778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.335608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:44.335911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:44.337280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:44.346378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 525/1025] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:44.348377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.453660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af737a-334f-492f-9a42-97c7ef4f6f01 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:44.464904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.488990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.489740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.490027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.490309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.491468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] RBAC: Adding query filter params (interface=public, service_id=270bfc1c7b8741528aea8c2008660efd, region_id=RegionOne) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:44.491907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:44.493876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b78fe1-d845-44ae-abdf-25d5d48df207 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:44.499798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 501/1026] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/endpoints?service_id=270bfc1c7b8741528aea8c2008660efd&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:44.505002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.537392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.551326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d83e7-310e-4dd7-b2ce-a13af3ce5065 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.551326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d83e7-310e-4dd7-b2ce-a13af3ce5065 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.551326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d83e7-310e-4dd7-b2ce-a13af3ce5065 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.551326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 502/1027] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:44.554790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.555085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.555367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.575884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:44.577095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:44.614246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe025-8f76-4ba9-bffd-cbde4f98a1d9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:44.646558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 503/1028] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:44.657696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.658395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.658693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.658978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.659642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:44.661023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:44.668974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 526/1029] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:42:44.680196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.791164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3546e8-76d0-44ba-91e5-356ae7a0309a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:44.794037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.831906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:44.832486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:44.832768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:44.833046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:44.852250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:44.857250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:44.858388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:44.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 504/1030] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:44.866807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:44.966663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:44.973875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f2d0e-bbfe-4ae4-bfda-8abe97fa805f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.009733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.010376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.010673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.010957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.011684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:45.012000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.013376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.026852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 527/1031] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:45.028557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.134759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe0f3-f400-4822-b91d-206bed42fecc None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.172654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.173352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.173639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.173921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.184598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.186049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-565a1edc-c362-4854-8c42-4111b901bf37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:45.201991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:45.203146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:45.204038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:45.204038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:45.204038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:45.204038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:45.206679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 505/1032] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:45.208994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.224042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1a5df-42c3-41f4-891f-a0f326687940 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.224384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1a5df-42c3-41f4-891f-a0f326687940 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.224609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1a5df-42c3-41f4-891f-a0f326687940 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.229599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 506/1033] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:45.231302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.231588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.231863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.251241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:45.252436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:45.264413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:45.323815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39dcda-cc16-48ba-8f85-7064616f3253 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.329695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 507/1034] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:45.354328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.360249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.360866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.361150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.361423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.362108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:45.362444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.363830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.376593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 528/1035] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:45.383088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.403062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:45.464594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb585d4-a78a-49d4-bd7e-85a7474523d0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.487390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487584-da73-4e19-92ad-4883e8c4033c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.499361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.500002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.500286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.500564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.501742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.503207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.506890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 508/1036] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:45.510940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.526242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.526881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.527168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.527447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.529180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:45.534738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.538771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:45.549223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:45.550237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:45.556856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 529/1037] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:45.558943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.628703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc2755-be58-4c23-a410-993b7c857405 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.645186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:45.670069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2f4f-dd48-44f2-8377-bd96d83961e1 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.670418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.670955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.671243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.671518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.672207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.673628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.707431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.708061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.708343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.708616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.709633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:45.710036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.711554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:45.731109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 530/1038] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:45.737189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:45.801655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 509/1039] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:42:45 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:42:45.856825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7816489-a4b6-461d-be23-857dd7bc6564 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:45.897672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:45.898518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:45.898518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:45.898802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:45.901851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:45.976759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:45.978150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.082093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 531/1040] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:46.088321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.160012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:46.193241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6fed-124e-4849-9685-852851f59477 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:46.229421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:46.229994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.230281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.230584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.242023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:46.243509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:46.258774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:46.259700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:46.260602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:46.260602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:46.260602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:46.260602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:46.262417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 510/1041] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:46.265720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.301772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:46.372227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf9f3-baf9-466a-a8a0-6447dbc57b7d None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:46.406607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:46.407226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.407510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.407789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.408487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:46.408795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:46.410159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.422394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 532/1042] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:46.427749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.529892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb380118-0197-4a8f-a057-c390483db0d5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:46.552923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:46.564905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:46.565552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.565847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.566120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.573237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:46.574711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-70bf0767-1687-4626-9335-27d084ed7794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:46.587253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:46.588166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:46.589057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 511/1043] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:46.595775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.692839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:46.700822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6720d-8f24-4cd5-82b2-5ee1a3b882ac None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:46.736155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:46.736776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.737059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.737340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.738087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:46.738803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:46.739917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.742316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ce71-884c-4738-b3d0-f4ef35c5812e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.742642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ce71-884c-4738-b3d0-f4ef35c5812e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.743094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ce71-884c-4738-b3d0-f4ef35c5812e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.744536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 512/1044] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:46.749541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.749858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.750211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.759127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 533/1045] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:46.760278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.770054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:46.771372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:46.845907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 513/1046] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:46.868602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:46.877683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30fabf-454d-4e9e-8985-16ca1cde3eb6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:46.920861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:46.921555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:46.921832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:46.922117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:46.922938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] RBAC: Adding query filter params (role.id=9b534570538f4486bca5138b598faa6b, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:46.923256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:46.924678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:46.930738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 534/1047] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=9b534570538f4486bca5138b598faa6b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:46.986539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc5093-ee0a-4d17-9a72-df0aa88bc1c4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:47.003060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:47.027512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:47.028181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.028469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.028857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] PATH_INFO: `/v3/services/image` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.029518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:47.031652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:47.042661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:47.043724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:47.044572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 514/1048] 10.222.0.85 () {56 vars in 1105 bytes} [Thu Jul 28 17:42:46 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:47.049955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:47.071824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:47.158719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948de24-d363-4503-90c4-6a296e7d3a5e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:47.193190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:47.194371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.194673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.194957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.195654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] RBAC: Adding query filter params (name=image) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:47.195967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:47.197439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:47.207558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 535/1049] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:47.208518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:47.312749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc533a6-91f6-40ee-b1e2-9c5b5659589e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:47.347689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:47.348335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.348624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.348905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.349589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] RBAC: Adding query filter params (type=image) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:47.349973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:47.351341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccadf42-0946-45f0-a6c5-9df4f0479b87 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:47.361236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 515/1050] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:47.362169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:47.449766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:47.468778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35a63d-837d-4242-8431-6e543424a9fc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:47.508198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:47.508826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.509113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.509393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.510121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:47.511523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44572ad-fb43-40f0-96e6-43487e6b2478 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:47.590626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 536/1051] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:42:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:42:47.886509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6574-bb45-42cc-8b8f-95f63ce938d8 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.887018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6574-bb45-42cc-8b8f-95f63ce938d8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.887096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6574-bb45-42cc-8b8f-95f63ce938d8 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.888752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 516/1052] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:47.894359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.894664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.894943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.914425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:47.915608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:47.993161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 537/1053] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:48.016165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6574-bb45-42cc-8b8f-95f63ce938d8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:48.064341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:48.125160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6574-bb45-42cc-8b8f-95f63ce938d8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:48.159447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:48.160092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.160381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.160667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.172187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:48.177042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:48.183925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:48.185017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 517/1054] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:48.187303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:48.250433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:48.288658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c7f0f-39ec-4582-a428-b0d6e2af4d30 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:48.323002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:48.323629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.323915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.324192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.325167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:48.325470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:48.327601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:48.338389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 538/1055] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:48.344121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:48.446665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:48.450155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03fff2-dabb-436e-9ab8-452b22671221 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:48.485614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:48.486256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.486567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.486847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] PATH_INFO: `/v3/users/1fd6d3c559ef44a79e080044f91f1dbc` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.500819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1fd6d3c559ef44a79e080044f91f1dbc)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:48.502264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:48.509275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 518/1056] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:42:48 2022] GET /identity/v3/users/1fd6d3c559ef44a79e080044f91f1dbc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:48.510540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:48.618368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21f4a6-2357-4aa2-a886-988bf2868af7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:48.645034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:48.654615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:48.658542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.658542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.658542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.727855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:48.729289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c70f-4a22-4ef5-b4c9-2ed4819c6396 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:48.842156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 539/1057] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:42:48.847971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:48.952456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ce845-9fe3-4c3c-96a4-fa66823bde5e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.952742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ce845-9fe3-4c3c-96a4-fa66823bde5e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.953018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ce845-9fe3-4c3c-96a4-fa66823bde5e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.958016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79bee1-b037-4015-b7e3-738a6a74b2b3 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:48.958478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 540/1058] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:48.960082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.960521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.960801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.979243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:48.980353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:48.995604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:48.996251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:48.996535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:48.996816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:48.997974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:48.999449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:49.001122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 519/1059] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:42:48 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:49.002560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.007761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.038994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.058387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 541/1060] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:49.077695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.126090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5dc47-d5fe-4d6d-a85f-b60525c30963 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.171253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.171893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.172457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.172745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.173804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3, user.id=1fd6d3c559ef44a79e080044f91f1dbc) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:49.174126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:49.176332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:49.182459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 520/1061] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/role_assignments?user.id=1fd6d3c559ef44a79e080044f91f1dbc&scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:49.183426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217bca5-5568-4038-be50-f282b79e3be5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.231920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.232501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.232789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.233071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.246560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:49.251774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:49.252764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:49.259211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 542/1062] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:49.262145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.363140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.374212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb05c6-3b5b-4ad5-a1e5-6f254535c6ac None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.419291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.419931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.420219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.420500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.421239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:49.421550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:49.422979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:49.437623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 521/1063] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:49.439252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.459387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.544706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0a1a2-7992-4963-9c41-167a3875794f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.579956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.580612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.580903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.581186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.592589 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:49.594068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-5bbe3956-f49e-4150-8393-9d712b318389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:49.609558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:49.610576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:49.611500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:49.611500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:49.611500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:49.611500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:49.614433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 543/1064] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:49.617460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.653217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15110a-fcde-4216-af36-2203aa6de1e5 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.653502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15110a-fcde-4216-af36-2203aa6de1e5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.653785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15110a-fcde-4216-af36-2203aa6de1e5 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.655289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 544/1065] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:49.661372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.661662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.661953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.683192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:49.684408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:49.735912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25217c2-5d3b-456e-a564-3afdb56cd234 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.757509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.764187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 545/1066] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:49.780239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.780834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.781119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.781397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.782101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:49.782438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:49.784518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:49.787880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.797810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 522/1067] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:49.804939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:49.821485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:49.910677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bf6df-6562-4964-8b66-6605feca74fd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.946932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9bcf8-020c-4f96-9ca6-eb6a6e819940 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:49.956095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.957409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.957526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.957741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.987977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:49.988657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.988904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.989182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.996923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:49.998343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:50.011393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:50.012626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:50.013622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 523/1068] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:50.018990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.034064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.038562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.040055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.100571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.133359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc097f7-22d9-440f-8caa-8f4df0eeb3d8 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.149239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 546/1069] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:50.155484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.170017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.170682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.171174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.171298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.172067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:50.172384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.173813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.192826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 524/1070] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:50.199535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.260578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.265093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c083a7-3ea0-4c99-9c6a-7f874d7bcca6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.310876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.311920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.312279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.312568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.315952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a002941-a2e1-49ce-8b48-9bd472b682c7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.323407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.324870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:50.341138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:50.342097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:50.343014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:50.343014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:50.343014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:50.343014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:50.343014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 547/1071] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:50.348089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.351075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.357715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.358385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.358674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.358960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.369641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.371155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:50.386535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:50.387624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:50.389133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:50.389133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:50.389133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:50.389133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:50.389133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 525/1072] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:50.394063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.462027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ccf3c-69af-4cbe-87b9-61808203bb35 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.515364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.516443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.516443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.516640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.517257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:50.517569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.517733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4582775-edf0-44ee-8fdd-8a47717419c5 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.519002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.542380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 548/1073] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:50.556279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.556932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.557222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.557504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.558228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:50.558601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.560026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.575142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 526/1074] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:50.580681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.594669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.696267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69deb6-5079-498c-9f15-f22924fbf35c None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.733623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.734703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.735345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.735622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.743755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.745220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:50.757760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:50.758704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:50.759588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 549/1075] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:50.765968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.814699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:50.879589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e1c5-4a8f-495d-91f0-8a845c5b2958 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.913669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.914306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.914622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.914908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.915669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:50.915979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.917687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16a028-556d-4b42-bec3-d7b779872662 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.932527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 527/1076] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:50.939327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:51.017466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:51.055843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10af983-cf84-4cf9-b88b-96d932935230 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:51.091640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:51.092652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.092652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.092896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/9b534570538f4486bca5138b598faa6b` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.095791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=1e396c6de22e485bb62fcecfd7f49243, role_id=9b534570538f4486bca5138b598faa6b)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:51.097000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bf172-ec08-4cd5-9455-1fb5fb22e979 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:51.156614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 550/1077] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:50 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/1e396c6de22e485bb62fcecfd7f49243/roles/9b534570538f4486bca5138b598faa6b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:51.238273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448f6f-214e-430b-b54f-3be6d44f733c None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.238878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448f6f-214e-430b-b54f-3be6d44f733c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.238878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448f6f-214e-430b-b54f-3be6d44f733c None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.240917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 528/1078] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:51.247093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.247382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.247685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.267666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:51.268906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:51.344548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 551/1079] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:51.368017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448f6f-214e-430b-b54f-3be6d44f733c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:51.475454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448f6f-214e-430b-b54f-3be6d44f733c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:51.477119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:51.510721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:51.511371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.511935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.512216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.523341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:51.528857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:51.529829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:51.535790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 529/1080] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:51.538532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:51.600791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:51.640605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0ba2c-b98d-488a-907f-b7e877ab74fe None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:51.676829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:51.677466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.677926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.678277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.679203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:51.679588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:51.681066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:51.691683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 552/1081] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:51.697010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:51.802809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f641ea6-9476-4d3b-adc0-bd6003a19cf4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:51.837230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:51.839253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.839253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.839253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] PATH_INFO: `/v3/users/1fd6d3c559ef44a79e080044f91f1dbc` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.840782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] RBAC: Authorizing `identity:get_user(user_id=1fd6d3c559ef44a79e080044f91f1dbc)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:51.842222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:51.844140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 530/1082] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:42:51 2022] GET /identity/v3/users/1fd6d3c559ef44a79e080044f91f1dbc => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:51.850310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:51.958206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416063f-906f-478c-bd91-e97ba376f133 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:51.981455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:51.993110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:51.993743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:51.994025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:51.994303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:51.995470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:51.996897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa762a-75fb-4e99-a22b-d26c38b16fff None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:52.003436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 553/1083] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:42:51 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:52.004880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:52.037200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:52.108744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b855e08-514d-40c8-affc-46a471ab58ea None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:52.142226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:52.142937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:52.143219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:52.143497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3/users/1fd6d3c559ef44a79e080044f91f1dbc/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:52.145435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] RBAC: Authorizing `identity:create_grant(project_id=c83a46b006cd40eb945ae4fdca8d84a3, user_id=1fd6d3c559ef44a79e080044f91f1dbc, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:52.147573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bd28d-752c-4599-822e-e5f9442251db None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:52.254653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 531/1084] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:42:52 2022] PUT /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3/users/1fd6d3c559ef44a79e080044f91f1dbc/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:42:53.204981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebe9f4-ed7c-4ecb-8af3-291f74d33330 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.205503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebe9f4-ed7c-4ecb-8af3-291f74d33330 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.205503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebe9f4-ed7c-4ecb-8af3-291f74d33330 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.206912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 554/1085] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:53.213445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.213812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.214165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.236288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:53.239903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:53.306215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 532/1086] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:53.329600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebe9f4-ed7c-4ecb-8af3-291f74d33330 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:53.436147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebe9f4-ed7c-4ecb-8af3-291f74d33330 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:53.470148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:53.470734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.471019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.471297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.482191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:53.487112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:53.488236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:53.489144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 555/1087] 10.222.0.85 () {56 vars in 1120 bytes} [Thu Jul 28 17:42:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:53.494518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:53.515816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:53.592634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6025a-4575-4631-ba49-c992ed2c1bcd None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:53.628550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:53.629172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.629454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.629729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.630474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:53.631313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:53.632814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:53.641301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 533/1088] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:42:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:53.647657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:53.704103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:53.757246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901d99-25cb-4c34-8498-4e5b1560a2bb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:53.792858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:53.793540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.793850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.794133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.808279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:53.810658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:53.826610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:53.827585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:53.828488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:53.828488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:53.828488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:53.828488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:53.828488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 556/1089] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:53.834069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:53.923783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:53.943856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a00ee5-6028-4f6f-a3a0-6e7ab3820510 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:53.978751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:53.979376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:53.979663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:53.979943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:53.980640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:53.980949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:53.982374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:53.994935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 534/1090] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:54.000589 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.104540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addde42f-6a9d-4419-aab2-6329bb52b533 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.106065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.140393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.141036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.141320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.141599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.149543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.151028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-eec08d66-fd63-4ca3-bee5-91129086ff9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:42:54.163182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:54.164164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:54.165132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 557/1091] 10.222.0.85 () {56 vars in 1093 bytes} [Thu Jul 28 17:42:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:42:54.170484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.289338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65afc0-8b99-49cd-b031-3ad8ed8ee8d1 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.289892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65afc0-8b99-49cd-b031-3ad8ed8ee8d1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.289892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65afc0-8b99-49cd-b031-3ad8ed8ee8d1 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.291235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 558/1092] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:54.296545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255924ad-25e1-47ec-9251-49b97de905da None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.296832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255924ad-25e1-47ec-9251-49b97de905da None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.297124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255924ad-25e1-47ec-9251-49b97de905da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.299050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88ba4-fbf6-45ae-b8ad-03afc1934b71 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.315704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-255924ad-25e1-47ec-9251-49b97de905da None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:54.316893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-255924ad-25e1-47ec-9251-49b97de905da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:54.334004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.334814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.334938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.335209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.336038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:54.336350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.337767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.352233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.355888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 535/1093] 10.222.0.85 () {56 vars in 1133 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:54.361478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.386528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 559/1094] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:42:54.409344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255924ad-25e1-47ec-9251-49b97de905da None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.463597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc668e6-a838-44ac-9b47-8fa45157aac7 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.497910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.498567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.498847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.499129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.499266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.507937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255924ad-25e1-47ec-9251-49b97de905da None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.542451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.543098 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.543383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.543662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.554864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:42:54.559734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:54.560762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:42:54.561702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 560/1095] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:54.564810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.566904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.567179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.568749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.671361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2080f-bc18-45f8-b12b-8e0446d4f7a6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.676242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 536/1096] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:54.683217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.710557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.711190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.711478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.711784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.712513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:54.712823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.714238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.723583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 561/1097] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:54.729597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.746691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.789696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:54.792932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aa83d-448f-44cf-8af8-561261e8f5f4 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.829277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.829929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.830209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.830525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.839825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c324e-3071-4443-8884-00c26d9a9106 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.841526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.843049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:42:54.858847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:54.859879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:54.861237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:42:54.861237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:42:54.861237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:42:54.861237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:54.861237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 537/1098] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:54.866102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.875576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:54.876232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:54.876520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:54.876806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] PATH_INFO: `/v3/users/1fd6d3c559ef44a79e080044f91f1dbc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:54.878015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=1fd6d3c559ef44a79e080044f91f1dbc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:54.879502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:54.881840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 562/1099] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/users/1fd6d3c559ef44a79e080044f91f1dbc => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:54.887453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:54.977081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c10a-bbd2-4640-be63-3c01801d0e55 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:54.980890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.010780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.011447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.011734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.011829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c040000-56a8-449b-96a5-b5ba0c2acbe7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.012015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.012708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:55.013014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.014415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.026224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 538/1100] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:55.031978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:55.051109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.070449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.071077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.071360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.071634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.146924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b909bf-1291-4b97-a974-6b4dcac1bd76 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.149691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.151270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.184111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.184751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.185027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.185305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.192159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.193612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:42:55.205140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:42:55.206254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:42:55.207308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 539/1101] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:42:55.212285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:55.238556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.259067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 563/1102] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:42:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:42:55.265139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:55.294274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.324696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e181b-a45f-42af-b3af-25867016e7e0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.363218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.363872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.364123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.364407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.365427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:55.365824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.367299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.371837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac425fa-36df-4821-908f-e0f348689c31 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.380077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 540/1103] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:55.386309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:55.410570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.411212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.411492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.411942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.413630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.415095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.416998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 564/1104] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:42:55 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:55.423117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:55.477406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.494906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7bfb9-e602-496b-95eb-079614c42e69 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.527762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:55.532176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a6dd-0ad2-45b9-8d34-0c3110774723 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:55.532714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] RBAC: Adding query filter params (role.id=9b534570538f4486bca5138b598faa6b, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=1e396c6de22e485bb62fcecfd7f49243) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.537588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6c2a-092e-4b7a-b7e9-409809227bc8 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.543454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 541/1105] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:55 2022] GET /identity/v3/role_assignments?user.id=1e396c6de22e485bb62fcecfd7f49243&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=9b534570538f4486bca5138b598faa6b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:55.575383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:55.576410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:55.576865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:55.577342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:55.578813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3, user.id=1fd6d3c559ef44a79e080044f91f1dbc) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:42:55.579142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:55.580686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563223f8-f6d1-40f1-b424-00323207faf3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:55.588486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 565/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:42:55 2022] GET /identity/v3/role_assignments?user.id=1fd6d3c559ef44a79e080044f91f1dbc&scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:42:57.689880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a946c-2cbc-4003-85f2-a65af52ac9cc None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:57.690437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a946c-2cbc-4003-85f2-a65af52ac9cc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:57.690437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a946c-2cbc-4003-85f2-a65af52ac9cc None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:57.691796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 542/1107] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:57.697112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:57.697526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:57.697670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:57.716691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:57.717907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:57.788686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 566/1108] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:42:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:42:57.813262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a946c-2cbc-4003-85f2-a65af52ac9cc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:57.918525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a946c-2cbc-4003-85f2-a65af52ac9cc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:57.953392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:57.953953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:57.954263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:57.954575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:57.955729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:57.957165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3411d-9af9-4f18-bfb3-143657d0f32e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:57.959478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 543/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:42:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:57.965285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:58.019486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:58.079322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62980-44e7-45bc-859e-5ceaee615b82 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:58.114046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:58.114713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:58.115207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:58.115647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:58.116341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:58.117772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafc285-67b5-4539-b1e7-4b72295aa7be None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:58.222220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 567/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:42:57 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:00.249238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d197-9edd-427d-b69e-9768bcf24bcc None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:00.249753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d197-9edd-427d-b69e-9768bcf24bcc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:00.249870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d197-9edd-427d-b69e-9768bcf24bcc None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:00.251257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 544/1111] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:00.257506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:00.257892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:00.258073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:00.277672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:00.278947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:00.346934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 568/1112] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:00.371328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254d197-9edd-427d-b69e-9768bcf24bcc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:00.435086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:00.473449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254d197-9edd-427d-b69e-9768bcf24bcc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:00.507223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:00.507787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:00.508067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:00.508343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] PATH_INFO: `/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:00.517148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] RBAC: Authorizing `identity:get_role(role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:00.518567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:00.519950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 545/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:00 2022] GET /identity/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:00.526568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:00.557718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:00.633992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d7ce-1120-479c-b23e-ee51ed67e0ac None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:00.668282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:00.668980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:00.669269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:00.669549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] PATH_INFO: `/v3/users/714a2978d124483ab87f1033df733894` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:00.683285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] RBAC: Authorizing `identity:get_user(user_id=714a2978d124483ab87f1033df733894)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:00.684779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:00.686451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 569/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:00 2022] GET /identity/v3/users/714a2978d124483ab87f1033df733894 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:00.692249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:00.802743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8707c-ebce-4492-a803-aa02d7885e50 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:00.831007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:00.836489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:00.842174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:00.842174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:00.842174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:00.904716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:00.906194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:01.009714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 546/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:01.019001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:01.111342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:01.124065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabca6c9-d431-4bbb-a31e-82f058c12993 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:01.160629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:01.161270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:01.161623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:01.161999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:01.163196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:01.164640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47757e-0142-498f-b389-534a6df790f5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:01.166332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 570/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:01 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:01.172821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:01.202905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:01.283465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f34da-3acd-4253-bdf2-13c4e30bbd05 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:01.316961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:01.317569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:01.318516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:01.318516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:01.319486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3, user.id=714a2978d124483ab87f1033df733894) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:01.319796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:01.321212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966fbd9-e65f-4e66-a210-24b6fc8b5b09 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:01.328122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 547/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:01 2022] GET /identity/v3/role_assignments?user.id=714a2978d124483ab87f1033df733894&scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:03.307513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ae4c-1b2d-473d-81dc-fedbed2427b3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:03.308085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ae4c-1b2d-473d-81dc-fedbed2427b3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:03.308085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ae4c-1b2d-473d-81dc-fedbed2427b3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:03.309447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 571/1118] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:03.315036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:03.315749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:03.315973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:03.334798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:03.336022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:03.405630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 548/1119] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:03.429843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508ae4c-1b2d-473d-81dc-fedbed2427b3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:03.544088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508ae4c-1b2d-473d-81dc-fedbed2427b3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:03.578125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:03.578732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:03.579018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:03.579300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] PATH_INFO: `/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:03.580752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] RBAC: Authorizing `identity:get_role(role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:03.582183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:03.584266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 572/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:03 2022] GET /identity/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:03.590119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:03.619030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:03.700204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7936d-aefb-4a57-9681-36aa958de0d6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:03.735800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:03.736448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:03.736729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:03.737001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] PATH_INFO: `/v3/users/714a2978d124483ab87f1033df733894` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:03.738154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] RBAC: Authorizing `identity:get_user(user_id=714a2978d124483ab87f1033df733894)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:03.739714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:03.741489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 549/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:03 2022] GET /identity/v3/users/714a2978d124483ab87f1033df733894 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:03.750077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:03.783083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:03.865983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70611198-331f-43ad-9a84-b654fc69b9f7 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:03.903128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:03.903779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:03.904069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:03.904353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:03.905534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:03.907062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967aced-a230-4f46-88d0-0add8849a64f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:03.908744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 573/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:03 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:03.916637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:04.021509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:04.029258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd293fa5-7779-4378-97a7-2d18ab866d2d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:04.063455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:04.064133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:04.064468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:04.064841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3/users/714a2978d124483ab87f1033df733894/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:04.068983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c83a46b006cd40eb945ae4fdca8d84a3, user_id=714a2978d124483ab87f1033df733894, role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:04.071930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0df2b3-ff0c-48f8-ae04-40ef827dd3e9 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:04.166236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 550/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:43:03 2022] PUT /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3/users/714a2978d124483ab87f1033df733894/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:43:06.158764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6e54-9c94-421d-a573-4d935bfacc92 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:06.159304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6e54-9c94-421d-a573-4d935bfacc92 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:06.159304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6e54-9c94-421d-a573-4d935bfacc92 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:06.160594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 574/1124] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:06.168179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:06.168681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:06.168782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:06.187753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:06.189001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:06.261786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 551/1125] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:06.285086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b6e54-9c94-421d-a573-4d935bfacc92 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:06.380119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:06.389623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b6e54-9c94-421d-a573-4d935bfacc92 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:06.425346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:06.425905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:06.426186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:06.426489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] PATH_INFO: `/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:06.428737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] RBAC: Authorizing `identity:get_role(role_id=fc5e3c6a371c460aba8e1bde5743fe12)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:06.430102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:06.431469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 575/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:06 2022] GET /identity/v3/roles/fc5e3c6a371c460aba8e1bde5743fe12 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:06.437460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:06.522157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:06.538142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48ee6-a812-4f82-ade8-3200bb21413f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:06.578082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:06.578835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:06.579120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:06.579400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] PATH_INFO: `/v3/users/714a2978d124483ab87f1033df733894` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:06.580670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] RBAC: Authorizing `identity:get_user(user_id=714a2978d124483ab87f1033df733894)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:06.582106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:06.584425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 552/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:06 2022] GET /identity/v3/users/714a2978d124483ab87f1033df733894 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:06.591063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:06.698717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9b53-af99-41ec-bcb1-9d00325707f9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:06.739401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:06.740063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:06.740352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:06.740633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:06.769553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:06.817730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:06.820507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:06.944673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 576/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:06.950766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:07.061225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b696da1-927e-473b-bdc1-5113214c735a None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:07.099528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:07.100318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:07.100605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:07.100886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] PATH_INFO: `/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:07.102136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] RBAC: Authorizing `identity:get_project(project_id=c83a46b006cd40eb945ae4fdca8d84a3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:07.103606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e3db-06af-4850-b098-2feb48f447b0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:07.105301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 553/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:06 2022] GET /identity/v3/projects/c83a46b006cd40eb945ae4fdca8d84a3 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:07.111561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:07.173888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:07.217853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809fde-f02f-4d82-8347-1038a199d121 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:07.259236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:07.259861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:07.260151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:07.260428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:07.261232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] RBAC: Adding query filter params (role.id=fc5e3c6a371c460aba8e1bde5743fe12, scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3, user.id=714a2978d124483ab87f1033df733894) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:07.261552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:07.263025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641233-6915-42d6-af01-6923ea26cf9e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:07.270147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 577/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:07 2022] GET /identity/v3/role_assignments?user.id=714a2978d124483ab87f1033df733894&scope.project.id=c83a46b006cd40eb945ae4fdca8d84a3&role.id=fc5e3c6a371c460aba8e1bde5743fe12 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:09.316455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5342ab-92d8-4a32-9d55-a9a256fd312b None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:09.317022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5342ab-92d8-4a32-9d55-a9a256fd312b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:09.317022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5342ab-92d8-4a32-9d55-a9a256fd312b None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:09.319292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 554/1131] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:09.324756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:09.325232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:09.325322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:09.344467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:09.345712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:09.419594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 578/1132] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:09.443084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5342ab-92d8-4a32-9d55-a9a256fd312b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:09.469141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:09.559813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5342ab-92d8-4a32-9d55-a9a256fd312b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:09.605239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:09.605813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:09.606098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:09.606412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:09.607613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:09.609057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93644b22-a5e5-4d19-8fc2-b0aa368c88c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:09.610745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 555/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:43:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:09.616804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:09.687026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:09.738025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82d496-08d2-4049-8a37-72ea39e1c34c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:09.776422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:09.777063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:09.777349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:09.777631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:09.778448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:09.779829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ff515-7eb4-4cff-9834-a2fc03b3b91a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:09.923522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 579/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:43:09 2022] POST /identity/v3/projects => generated 311 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:11.984846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b3ac2-6a99-4816-9b9e-34b2b1dbde6e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:11.985451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b3ac2-6a99-4816-9b9e-34b2b1dbde6e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:11.985451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b3ac2-6a99-4816-9b9e-34b2b1dbde6e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:11.986795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 556/1135] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:11.993004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:11.993483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:11.993570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:12.012897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:12.014104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:12.081677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 580/1136] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:43:12.105925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b3ac2-6a99-4816-9b9e-34b2b1dbde6e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:12.214655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:12.228420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b3ac2-6a99-4816-9b9e-34b2b1dbde6e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:12.265609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:12.266256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:12.266575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:12.266858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:12.268067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:12.269505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed507f-05ce-4d0b-9150-1c654bb79f67 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:12.271508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 557/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:43:12.277824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:12.315974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:12.384512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486168b-d1bb-4833-bfc4-2e31154f1890 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:12.422061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:12.422721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:12.423003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:12.423282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:12.424012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:12.425465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8467b-7058-4b6e-b958-69d7695d26b3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:12.528275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 581/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:43:12 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:14.627914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79d12-7ab9-4d05-921f-6fcf34dadf29 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:14.628448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79d12-7ab9-4d05-921f-6fcf34dadf29 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:14.628448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee79d12-7ab9-4d05-921f-6fcf34dadf29 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:14.629925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 558/1139] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:14.635364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:14.635829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:14.635953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:14.654458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:14.655648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:14.724789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 582/1140] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:14.749133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee79d12-7ab9-4d05-921f-6fcf34dadf29 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:14.855411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee79d12-7ab9-4d05-921f-6fcf34dadf29 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:14.901278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:14.902096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:14.902533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:14.902911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:14.913460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:14.918632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:14.919707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:14.920836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 559/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:14.926004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:14.950887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:15.032939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b7f1a-5381-4f4f-8ab4-ef24334cf516 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:15.069529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:15.070161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:15.070476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:15.070756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:15.071483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:15.071794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:15.073173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:15.081488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 583/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:15.087383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:15.105697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:15.188984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448d1f9-1d05-4b13-b974-58e09734d781 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:15.223809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:15.224590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:15.224979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:15.225361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] PATH_INFO: `/v3/users/764b5ef29fe04f36bded19fc15e18b97` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:15.239766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] RBAC: Authorizing `identity:get_user(user_id=764b5ef29fe04f36bded19fc15e18b97)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:15.241320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:15.243170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 560/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:15 2022] GET /identity/v3/users/764b5ef29fe04f36bded19fc15e18b97 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:15.249094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:15.345212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:15.361672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f59e6-6aff-4802-97e4-1521e33ce057 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:15.395714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:15.396330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:15.396621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:15.396901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:15.463714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:15.465097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:15.547939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:15.564897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 584/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:15.571330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:15.651306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:15.682507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bcc39-a253-45c0-a374-26da4f6e6f26 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:15.724206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:15.724762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:15.725047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:15.725319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] PATH_INFO: `/v3/projects/e86d959150c141eaba3129a2cfe2aee2` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:15.726500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] RBAC: Authorizing `identity:get_project(project_id=e86d959150c141eaba3129a2cfe2aee2)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:15.727921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff7dd-e0c7-4d24-9bdb-d671cb39c6f5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:15.729602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 561/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:15 2022] GET /identity/v3/projects/e86d959150c141eaba3129a2cfe2aee2 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:15.735917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:15.844283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4169b96-ad1f-4cb9-8b09-5ca7ada96962 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:15.880985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:15.881613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:15.881904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:15.882225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:15.883080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=e86d959150c141eaba3129a2cfe2aee2, user.id=764b5ef29fe04f36bded19fc15e18b97) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:15.883397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:15.884843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791ffa4-1f53-4428-8000-64a5a8d8ff1e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:15.890523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 585/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:15 2022] GET /identity/v3/role_assignments?user.id=764b5ef29fe04f36bded19fc15e18b97&scope.project.id=e86d959150c141eaba3129a2cfe2aee2&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:17.917158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142010f8-5188-4a89-9fad-21ec345b460d None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:17.917688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142010f8-5188-4a89-9fad-21ec345b460d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:17.917688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142010f8-5188-4a89-9fad-21ec345b460d None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:17.919105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 562/1147] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:17.924671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:17.925146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:17.925237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:17.944939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:17.946148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:17.995076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:18.021445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 586/1148] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:18.044521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142010f8-5188-4a89-9fad-21ec345b460d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:18.146730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142010f8-5188-4a89-9fad-21ec345b460d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:18.184157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:18.184796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:18.185076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:18.185363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:18.195664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:18.200546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:18.201750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:18.202932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 563/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:18.207996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:18.309846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0743223-4ad7-4e05-96df-47472d9b9f80 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:18.343987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:18.345763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:18.345763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:18.345763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:18.346313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:18.346313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:18.347600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:18.356358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 587/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:18.362197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:18.366485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:18.474112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4309-d8ca-4b15-9157-b1fa0074e056 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:18.508136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:18.508762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:18.509042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:18.509315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] PATH_INFO: `/v3/users/764b5ef29fe04f36bded19fc15e18b97` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:18.510504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] RBAC: Authorizing `identity:get_user(user_id=764b5ef29fe04f36bded19fc15e18b97)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:18.511924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:18.513543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 564/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:18 2022] GET /identity/v3/users/764b5ef29fe04f36bded19fc15e18b97 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:18.519385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:18.561452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:18.620753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0684ff-e8c5-48b5-94c8-5eae91822eb9 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:18.653773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:18.654426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:18.654710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:18.654987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] PATH_INFO: `/v3/projects/e86d959150c141eaba3129a2cfe2aee2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:18.656122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] RBAC: Authorizing `identity:get_project(project_id=e86d959150c141eaba3129a2cfe2aee2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:18.657555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fa9dd-50fd-41be-bbf5-4649da35e6fc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:18.659230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 588/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:18 2022] GET /identity/v3/projects/e86d959150c141eaba3129a2cfe2aee2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:18.666333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:18.739201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:18.770610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dd7db-9d0a-4da5-acf4-eb52e5c33647 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:18.807027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:18.807693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:18.807976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:18.808250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] PATH_INFO: `/v3/projects/e86d959150c141eaba3129a2cfe2aee2/users/764b5ef29fe04f36bded19fc15e18b97/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:18.810174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e86d959150c141eaba3129a2cfe2aee2, user_id=764b5ef29fe04f36bded19fc15e18b97, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:18.811624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f795da0-79ea-4f8f-86d0-66288fbec8d2 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:18.932852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 565/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:43:18 2022] PUT /identity/v3/projects/e86d959150c141eaba3129a2cfe2aee2/users/764b5ef29fe04f36bded19fc15e18b97/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:43:20.930795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd3a9-15e3-4848-843a-cb03c5c3961f None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:20.931346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd3a9-15e3-4848-843a-cb03c5c3961f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:20.931346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd3a9-15e3-4848-843a-cb03c5c3961f None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:20.932703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 589/1154] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:20.938044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:20.938504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:20.938706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:20.957810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:20.959260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:21.030373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 566/1155] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:21.054829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd3a9-15e3-4848-843a-cb03c5c3961f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:21.159999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd3a9-15e3-4848-843a-cb03c5c3961f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:21.171233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:21.196625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:21.197260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:21.197546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:21.197827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:21.208770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:21.213682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:21.214772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:43:21.215689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 590/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:21.221287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:21.281391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:21.323954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f9b0-c17a-452d-be4d-e05c2e283ffe None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:21.359214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:21.359887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:21.360175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:21.360457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:21.361184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:21.361495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:21.362917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:21.371904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 567/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:21.378006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:21.489823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add7955-802c-4591-b318-1e905da6c39f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:21.526211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:21.526893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:21.527178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:21.527459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] PATH_INFO: `/v3/users/764b5ef29fe04f36bded19fc15e18b97` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:21.528648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] RBAC: Authorizing `identity:get_user(user_id=764b5ef29fe04f36bded19fc15e18b97)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:21.530097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:21.531755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 591/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:21 2022] GET /identity/v3/users/764b5ef29fe04f36bded19fc15e18b97 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:21.537647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:21.646025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cedc2-747d-4b26-9956-95f1479352c1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:21.676888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:21.689327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:21.689953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:21.690240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:21.697330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:21.762521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:21.763921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:21.869918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 568/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:21.875814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:21.912017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:21.978211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9dd3-e52e-4ebd-8588-5f28ad6a1d38 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:22.020515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:22.021069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:22.021347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:22.021619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] PATH_INFO: `/v3/projects/e86d959150c141eaba3129a2cfe2aee2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:22.022784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=e86d959150c141eaba3129a2cfe2aee2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:22.024212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922144ec-3432-4ad3-9421-28e0e0b0c6a5 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:22.025849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 592/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:21 2022] GET /identity/v3/projects/e86d959150c141eaba3129a2cfe2aee2 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:22.032271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:22.057955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:22.134318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731a52d-b852-430d-b138-0554a76c6c9d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:22.175257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:22.175881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:22.176827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:22.177225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:22.178185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=e86d959150c141eaba3129a2cfe2aee2, user.id=764b5ef29fe04f36bded19fc15e18b97) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:22.178537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:22.180010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e9187-fed8-49f0-b74f-2b38ebbf20c5 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:22.186373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 569/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:22 2022] GET /identity/v3/role_assignments?user.id=764b5ef29fe04f36bded19fc15e18b97&scope.project.id=e86d959150c141eaba3129a2cfe2aee2&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:24.240695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5be5a-0be3-481b-8067-cace4aa23b33 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:24.241277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5be5a-0be3-481b-8067-cace4aa23b33 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:24.241277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5be5a-0be3-481b-8067-cace4aa23b33 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:24.242634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 593/1162] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:24.248061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:24.248583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:24.248689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:24.267777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:24.268963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:24.336639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 570/1163] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:24.358623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5be5a-0be3-481b-8067-cace4aa23b33 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:24.467647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5be5a-0be3-481b-8067-cace4aa23b33 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:24.470401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:24.506710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:24.507251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:24.507530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:24.508128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:24.577914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:24.579402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:24.681264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 594/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:24.690891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:24.710477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:24.794434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96741e82-feb4-4cdb-91d6-a572a645ec42 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:24.838922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:24.839119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:24.839408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:24.839702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:24.847023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:24.848866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-868ad6e6-dff2-4e7b-b65e-5d98a0388c19 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:24.860774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:24.861989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:24.863102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 571/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:43:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:24.868763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:24.872069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:24.981294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96825f9-4d5c-4f56-b402-d3b374e3537f None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:25.016508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:25.017133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:25.017418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:25.017699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:25.018429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:25.018744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:25.020218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acae7ce-2ea7-429f-93a9-c306d3ebce59 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:25.028039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 595/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Thu Jul 28 17:43:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:27.044108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb18db-01bf-48b7-9074-9c6f8ae97d6b None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:27.044671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb18db-01bf-48b7-9074-9c6f8ae97d6b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:27.044671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb18db-01bf-48b7-9074-9c6f8ae97d6b None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:27.046021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 572/1167] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:27.052172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e797-85d2-4728-860f-552629cdab8b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:27.052724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e797-85d2-4728-860f-552629cdab8b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:27.052724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e797-85d2-4728-860f-552629cdab8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:27.072671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6334e797-85d2-4728-860f-552629cdab8b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:27.073900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6334e797-85d2-4728-860f-552629cdab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:27.135935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:27.141504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 596/1168] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:27.164446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb18db-01bf-48b7-9074-9c6f8ae97d6b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:27.240383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:27.263713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb18db-01bf-48b7-9074-9c6f8ae97d6b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:27.298553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:27.299155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:27.299433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:27.299706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:27.300418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:27.302012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e5b7a-9bfd-4a6f-9db7-5778a369f608 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:27.413976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 573/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Thu Jul 28 17:43:27 2022] POST /identity/v3/domains => generated 267 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:29.446906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ab9b-7d94-4b15-ab61-e29ed2860252 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:29.447468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ab9b-7d94-4b15-ab61-e29ed2860252 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:29.447468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ab9b-7d94-4b15-ab61-e29ed2860252 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:29.448847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 597/1170] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:29.454284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:29.454748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:29.454877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:29.473428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:29.474680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:29.547142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 574/1171] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:29.570897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92ab9b-7d94-4b15-ab61-e29ed2860252 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:29.678892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92ab9b-7d94-4b15-ab61-e29ed2860252 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:29.715590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:29.716160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:29.716444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:29.716724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] PATH_INFO: `/v3/domains/d18cc102fbf84eeea6cc20f52abf4067` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:29.728302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d18cc102fbf84eeea6cc20f52abf4067)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:29.729781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40146b4-081d-4670-9ab8-20e4ed262054 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:29.731514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 598/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Thu Jul 28 17:43:29 2022] GET /identity/v3/domains/d18cc102fbf84eeea6cc20f52abf4067 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:29.737536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:29.807294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:29.844228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45328902-993a-46b6-8c3a-258b91e716d1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:29.880213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:29.880841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:29.881125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:29.881409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:29.882104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:29.883569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84fb0a-4565-4016-b639-6520709a8354 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:29.980629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 575/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:43:29 2022] POST /identity/v3/projects => generated 361 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:32.098179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99250-f8ce-473a-91e4-7d4caeb47b86 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:32.098777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99250-f8ce-473a-91e4-7d4caeb47b86 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:32.098777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99250-f8ce-473a-91e4-7d4caeb47b86 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:32.100063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 599/1174] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:32.105350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:32.105805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:32.105911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:32.125885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:32.127127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:32.202656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 576/1175] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:32.227419 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa99250-f8ce-473a-91e4-7d4caeb47b86 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:32.292599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:32.338046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa99250-f8ce-473a-91e4-7d4caeb47b86 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:32.376140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:32.376695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:32.376974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:32.377248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] PATH_INFO: `/v3/domains/d18cc102fbf84eeea6cc20f52abf4067` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:32.378421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=d18cc102fbf84eeea6cc20f52abf4067)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:32.380801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfe96e-868b-4f97-b49b-133e1e69aa4d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:32.382414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 600/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Thu Jul 28 17:43:32 2022] GET /identity/v3/domains/d18cc102fbf84eeea6cc20f52abf4067 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:32.388367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:32.438206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:32.488619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d0c4-b3fe-47d7-bfc3-187b6b880506 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:32.523004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:32.523623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:32.523907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:32.524187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:32.524865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:32.526291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50af32-546f-454a-8fb6-858f0e5edaac None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:32.647108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 577/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:43:32 2022] POST /identity/v3/users => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:43:34.705390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6573e2-bc98-41be-b3ab-67ce9ee01ccc None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:34.705959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6573e2-bc98-41be-b3ab-67ce9ee01ccc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:34.705959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6573e2-bc98-41be-b3ab-67ce9ee01ccc None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:34.707987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 601/1178] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:34.713528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:34.713996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:34.714113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:34.732409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:34.733674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:34.810969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 578/1179] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:34.836068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6573e2-bc98-41be-b3ab-67ce9ee01ccc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:34.943647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6573e2-bc98-41be-b3ab-67ce9ee01ccc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:34.970046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:34.978934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:34.984701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:34.984701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:34.984701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:34.992063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:34.997068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:34.998017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:43:34.998915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 602/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:35.005244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:35.040561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:35.111540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b9e3f-48ac-4304-a688-18d898ed65d3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:35.148793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:35.149413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:35.149693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:35.149980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:35.150735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:35.151038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:35.152436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:35.162722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 579/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:35.169200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:35.310496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfcdd6-da16-4e7c-bd21-2f336db9ecfb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:35.346613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:35.347877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:35.348376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:35.348799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] PATH_INFO: `/v3/users/976877e30b004b7f87785372aecf93f3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:35.363000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] RBAC: Authorizing `identity:get_user(user_id=976877e30b004b7f87785372aecf93f3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:35.364791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:35.366499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 603/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:35 2022] GET /identity/v3/users/976877e30b004b7f87785372aecf93f3 => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:43:35.372408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:35.475186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:35.481070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacea116-b97e-4429-b99e-e92d877f190d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:35.525980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:35.526635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:35.526917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:35.527193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:35.597204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:35.598925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:35.692812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:35.699426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 580/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:35.705509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:35.759018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:35.812503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2950d61-eb82-4f7f-b15d-d56d16df9b59 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:35.848747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:35.849334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:35.849620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:35.849897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] PATH_INFO: `/v3/projects/be6bc52fda5846c8b3058f3b26982c9d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:35.851084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] RBAC: Authorizing `identity:get_project(project_id=be6bc52fda5846c8b3058f3b26982c9d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:35.852526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59274ce5-d29e-437f-b931-cdb4b233a491 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:35.854194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 604/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:35 2022] GET /identity/v3/projects/be6bc52fda5846c8b3058f3b26982c9d => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:35.860418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:35.960609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a87760-dd23-417d-bf91-3420193bef56 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:35.996009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:35.996633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:35.996920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:35.997200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:35.998004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=be6bc52fda5846c8b3058f3b26982c9d, user.id=976877e30b004b7f87785372aecf93f3) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:35.998448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:35.999896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c296677-17ca-4d14-8d0d-3873de4a5b85 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:36.006384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 581/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:35 2022] GET /identity/v3/role_assignments?user.id=976877e30b004b7f87785372aecf93f3&scope.project.id=be6bc52fda5846c8b3058f3b26982c9d&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:37.970661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32590-e5cb-44c2-9eb0-f211307b1256 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:37.971183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32590-e5cb-44c2-9eb0-f211307b1256 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:37.971300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32590-e5cb-44c2-9eb0-f211307b1256 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:37.973131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 605/1186] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:37.978545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:37.978970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:37.979109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:37.998072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:37.999290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:38.054806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:38.065264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 582/1187] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:38.088376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e32590-e5cb-44c2-9eb0-f211307b1256 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:38.190894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e32590-e5cb-44c2-9eb0-f211307b1256 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:38.203768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:38.224403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:38.225030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:38.225310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:38.225586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:38.235299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:38.240932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:38.242091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:43:38.243255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 606/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:38.248287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:38.345849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d8ba-bef3-4913-ba09-862a83285933 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:38.380831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:38.381505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:38.381815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:38.382096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:38.382843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:38.383155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:38.384530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:38.392571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 583/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:38.398529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:38.497142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f412c8-0104-4c85-a8ba-979c002c95cc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:38.529507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:38.530130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:38.530441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:38.530719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] PATH_INFO: `/v3/users/976877e30b004b7f87785372aecf93f3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:38.531880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] RBAC: Authorizing `identity:get_user(user_id=976877e30b004b7f87785372aecf93f3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:38.533308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:38.535424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 607/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:38 2022] GET /identity/v3/users/976877e30b004b7f87785372aecf93f3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:38.541392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:38.593541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:38.640446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd4fc9-97c9-46b7-adb8-577fcef9047d None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:38.673057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:38.673688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:38.673975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:38.674255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] PATH_INFO: `/v3/projects/be6bc52fda5846c8b3058f3b26982c9d` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:38.676148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] RBAC: Authorizing `identity:get_project(project_id=be6bc52fda5846c8b3058f3b26982c9d)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:38.677720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de59d7-b7df-47cf-a413-2ec1438053ab None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:38.679412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 584/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:38 2022] GET /identity/v3/projects/be6bc52fda5846c8b3058f3b26982c9d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:38.685421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:38.726303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:38.781529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414a8e3-1072-45eb-b037-68627cb00e70 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:38.815481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16614017-2810-460a-afdf-58a94861319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:38.816248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16614017-2810-460a-afdf-58a94861319b None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:38.816524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16614017-2810-460a-afdf-58a94861319b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:38.816800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16614017-2810-460a-afdf-58a94861319b None admin] PATH_INFO: `/v3/projects/be6bc52fda5846c8b3058f3b26982c9d/users/976877e30b004b7f87785372aecf93f3/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:38.818751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16614017-2810-460a-afdf-58a94861319b None admin] RBAC: Authorizing `identity:create_grant(project_id=be6bc52fda5846c8b3058f3b26982c9d, user_id=976877e30b004b7f87785372aecf93f3, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:38.820184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16614017-2810-460a-afdf-58a94861319b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:38.901410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 608/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:43:38 2022] PUT /identity/v3/projects/be6bc52fda5846c8b3058f3b26982c9d/users/976877e30b004b7f87785372aecf93f3/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:43:40.885719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbb7d-a393-4f64-baff-fff6f50ef1a5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:40.886239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbb7d-a393-4f64-baff-fff6f50ef1a5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:40.886330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbbb7d-a393-4f64-baff-fff6f50ef1a5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:40.887630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 585/1193] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:43:40.895680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:40.896159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:40.896278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:40.914165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:43:40.915628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:43:40.980019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 609/1194] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:43:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:43:41.002647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbbb7d-a393-4f64-baff-fff6f50ef1a5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.104287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbbb7d-a393-4f64-baff-fff6f50ef1a5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:41.123682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:41.137478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:41.138262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:41.138427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:41.138704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:41.148643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:43:41.153499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:43:41.154614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:43:41.155461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 586/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:43:41.161299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.192025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:41.259221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8731e2-1136-4406-9bbb-adbcbeab06f0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:41.292892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:41.293501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:41.293783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:41.294057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:41.294796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:41.295101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:41.296466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:41.304844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 610/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:41.310817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.408110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6ec04-c243-4035-a9ac-fe3eb7167cc2 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:41.442004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:41.442660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:41.442945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:41.443227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] PATH_INFO: `/v3/users/976877e30b004b7f87785372aecf93f3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:41.444398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] RBAC: Authorizing `identity:get_user(user_id=976877e30b004b7f87785372aecf93f3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:41.445825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:41.447449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 587/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/users/976877e30b004b7f87785372aecf93f3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:41.453372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.542185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:41.550672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dfe0d-ca1d-47bc-97ea-2be1e77daf8a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:41.584011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:41.584624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:41.584908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:41.585186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:41.650690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:41.652146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:41.739116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:41.748140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 611/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:43:41.754641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.803943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:41.853162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659dfda8-16e8-43ec-89ce-b667696500b1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:41.888235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:41.888874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:41.889161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:41.889441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] PATH_INFO: `/v3/projects/be6bc52fda5846c8b3058f3b26982c9d` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:41.890630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] RBAC: Authorizing `identity:get_project(project_id=be6bc52fda5846c8b3058f3b26982c9d)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:41.892105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8d74-ea2c-47e2-b3f1-deea109ffc83 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:41.893799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 588/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/projects/be6bc52fda5846c8b3058f3b26982c9d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:43:41.900011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:43:41.998215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8794ed9-8ded-4aa1-a681-2b89094ddcc8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:43:42.033128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:43:42.033742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:43:42.034029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:43:42.034316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:43:42.035207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=be6bc52fda5846c8b3058f3b26982c9d, user.id=976877e30b004b7f87785372aecf93f3) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:43:42.035523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:43:42.036961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dbe8d-37df-4a65-804a-b94ba4f99b62 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:43:42.043011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 612/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:43:41 2022] GET /identity/v3/role_assignments?user.id=976877e30b004b7f87785372aecf93f3&scope.project.id=be6bc52fda5846c8b3058f3b26982c9d&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:43:51.884060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=42 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:52.040355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=72 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:56.188303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226a53-f791-46ae-bb48-cb2e54023d36 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.188881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226a53-f791-46ae-bb48-cb2e54023d36 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.189178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226a53-f791-46ae-bb48-cb2e54023d36 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.191275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 589/1201] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:44:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:56.196495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.196909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.197055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.215182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:56.216390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:56.284019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 613/1202] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:44:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:44:56.308719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226a53-f791-46ae-bb48-cb2e54023d36 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:56.412262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226a53-f791-46ae-bb48-cb2e54023d36 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.448987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.449554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.449835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.450106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.460629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.462072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:44:56.476900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:44:56.479063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:44:56.480953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:44:56.480953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:44:56.480953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:44:56.480953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:44:56.480953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 590/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:44:56.486294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:56.583875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadca1f-bc78-4d56-ba92-b580973da277 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.615221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:56.622593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.623220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.623507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.623837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.624528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:56.624835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.626218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83936869-8d16-4839-a710-ba37db424cdd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.637380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 614/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:56.643677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:56.697796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:56.745282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c86dd3-8c43-4aee-936f-8ce7f60843d6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.781797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.782454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.782742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.783019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.783719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.785186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759536a-de82-4bee-8a42-8dfd63644f4f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.889929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 591/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Thu Jul 28 17:44:56 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:59.017809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf872-0be0-4d14-aadd-60bfa3f8af26 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.018390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf872-0be0-4d14-aadd-60bfa3f8af26 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.018826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf872-0be0-4d14-aadd-60bfa3f8af26 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.020867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 615/1206] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:44:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:59.026060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.026520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.026634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.047299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:59.048501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:59.119676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 592/1207] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:44:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:44:59.143287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf872-0be0-4d14-aadd-60bfa3f8af26 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.255824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf872-0be0-4d14-aadd-60bfa3f8af26 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.296883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.297521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.297805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.298080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.308934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:44:59.314186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:44:59.317051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:44:59.317997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 616/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:44:59.321694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.363668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.424886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cc6ef-7a28-48ca-93be-5152c4718679 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.464730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.465356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.465645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.465928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.466683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:59.466999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.468694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.478156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 593/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:59.484162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.540202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.591724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc007f-2b65-4434-8e1b-e0f46db4a614 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.625295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.626165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.626485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.626820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.637946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.640006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:44:59.654442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:44:59.655886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:44:59.657507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:44:59.657507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:44:59.657507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:44:59.657507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:44:59.657507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 617/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:44:59.662157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.767543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.770500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510f87f-4370-4819-8322-afdfaefed017 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.806907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.807526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.807809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.808090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.808878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:59.809175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.810552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.822921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 594/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:44:59.829139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.939826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a06fb3-4f43-42be-a6be-fae9cfe8f0be None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.944197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.982718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.984634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.984919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.985198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.993075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.994563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:00.007214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:00.008386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:00.009360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 618/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:00.014942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.130042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513210e-2293-419b-96b7-4292d755b331 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:00.163611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:00.164223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:00.164504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:00.164780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:00.165568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:00.165877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:00.167418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:00.184264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 595/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:00.190737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.292570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc0a6c-503b-4da7-91ae-811dbaa2c309 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:00.329254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:00.329883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:00.330170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:00.330480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:00.361627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:00.402633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:00.403840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:00.511388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 619/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:45:00.518101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.537272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:00.617754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2232a12-2226-44ad-9548-0c50cc348a9f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:00.652345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:00.653163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:00.653396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:00.653671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:00.665258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:00.666746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:00.683806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:00.685182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:00.687189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:00.687189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:00.687189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:00.687189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:00.687189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 596/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:00.691745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.755260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:00.804533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e73fb1-2988-4a89-a849-65a5625b39fb None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:00.840416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:00.841045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:00.841332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:00.841612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:00.842341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:00.842745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:00.844138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:00.856293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 620/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:00.863641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.935361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:00.967267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7f853-c1f0-4985-adc2-362925c1f575 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:01.005252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:01.005888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:01.006175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:01.006486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:01.014904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:01.016408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:01.027672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:01.028747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:01.030048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 597/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:01.035001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:01.141427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172fed8-d78c-4f91-b465-9348e617f6ec None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:01.163026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:01.174915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:01.175537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:01.175829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:01.176104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:01.176862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:01.177178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:01.178645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfc299-6310-4fa7-ac23-70ecbc429376 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:01.192936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 621/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:01.197564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:01.305043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711789d9-89dc-4337-b969-0c77b5d2381b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:01.334555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:01.340314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:01.340931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:01.341216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:01.341495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:01.342430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=90c8ff3469974d15a2a3ec8d7af9fe07) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:01.342920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:01.344354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306abba-5476-44f3-bb70-802bee513f1a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:01.349759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 598/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:45:01 2022] GET /identity/v3/role_assignments?user.id=90c8ff3469974d15a2a3ec8d7af9fe07&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:03.354302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c11540-0aa0-4d62-8e8f-367c9c9d1386 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.354937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c11540-0aa0-4d62-8e8f-367c9c9d1386 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.354937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c11540-0aa0-4d62-8e8f-367c9c9d1386 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.356223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 622/1220] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:03.364194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.364194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.364194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.381528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:03.382778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:03.450622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 599/1221] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:03.473551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c11540-0aa0-4d62-8e8f-367c9c9d1386 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:03.578462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c11540-0aa0-4d62-8e8f-367c9c9d1386 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:03.612229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:03.612785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.613070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.613344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.623890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:45:03.628767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:03.629881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:03.630985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 623/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:45:03.636085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:03.746830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971657-c8e6-4f16-a7f8-1581f6b3a9c4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:03.771076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:03.782707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:03.783328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.783613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.783894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.784615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] RBAC: Adding query filter params (name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:03.784993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:03.786473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:03.798094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 600/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:45:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:03.804113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:03.850153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:03.910591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b6283-d778-4b9a-b310-9c6e6c842fd3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:03.946606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:03.947250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.947534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.947814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.958614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:03.960109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:03.977425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:03.978570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:03.979881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:03.979881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:03.979881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:03.979881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:03.979881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 624/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:45:03.985469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.091057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d085c55-1d88-4dea-b9a5-4d7ceff3a464 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.125806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.126449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.126738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.127018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.127705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:04.128010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.129368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.141392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 601/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:04.147229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.243825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:04.250732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14465a-9c85-4a8f-9f72-ac770f3ac3f3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.287542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.288184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.288474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.288737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.297593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.299061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:04.310180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:04.311407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:04.317270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 625/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:04.318023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.368459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:04.424593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7b445-80ef-43b2-8568-ce7a72a0057c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.463376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.464095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.464384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.464706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.465629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:04.465842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.468124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.489504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 602/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:04.497802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.608489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eaea9-f120-4b00-bbd2-616e3dce3132 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.645781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.646481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.646839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.647117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.658746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.665307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:04.675881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:04.677065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:04.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:04.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:04.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:04.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:04.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 626/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:04.683095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.791640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e928d1-0e40-4be9-9314-568f4c5c1c80 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.794298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:04.831532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.832165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.832452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.832732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.833436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:04.833748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.835230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.848863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 603/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:04.853923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.863810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:04.962369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923763-f0bc-4620-9d7f-c358d26c3d0e None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.997593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.998931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.999241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.999523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.008497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.011170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:05.025427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:05.026583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:05.027812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 627/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:05.033261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.141323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e74992-6a6e-4aeb-a81a-c2fa687dff35 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.178139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.178805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.179206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.179492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.180253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:05.180565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.182127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918f759-e62a-48de-b462-af98c9048e1a None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.196652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 604/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:05.203168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.279334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:05.324677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332f54-6daa-46ab-9d74-70c6e98808ad None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.369593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.370301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.370642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.370925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/90c8ff3469974d15a2a3ec8d7af9fe07/roles/cf2a61f9748b49fbac555810a3da89da` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.390799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=90c8ff3469974d15a2a3ec8d7af9fe07, role_id=cf2a61f9748b49fbac555810a3da89da)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.392328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a2d10-0068-4aee-812f-e0a88f2f5a0d None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.470341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 628/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:45:05 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/90c8ff3469974d15a2a3ec8d7af9fe07/roles/cf2a61f9748b49fbac555810a3da89da => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:07.558124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c80c7b-edf3-4d30-9748-677c3f4ad98e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.558956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c80c7b-edf3-4d30-9748-677c3f4ad98e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.559136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c80c7b-edf3-4d30-9748-677c3f4ad98e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.561293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 605/1233] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:07.566538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.566988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.567210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.586071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:07.593770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.656732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 629/1234] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:07.681004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c80c7b-edf3-4d30-9748-677c3f4ad98e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.708244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.787117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c80c7b-edf3-4d30-9748-677c3f4ad98e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.822198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.822766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.823058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.823337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.834998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:45:07.839892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:07.841089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:07.847285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 606/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Thu Jul 28 17:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:45:07.848144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.908310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.960462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16132cf5-ffc7-445a-9a1b-689da05b8bf3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.995466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.996130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.996414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.996693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.997413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:07.997722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.999661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.007868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 630/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Thu Jul 28 17:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:45:08.015201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.104486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:08.121032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59e70-c6f1-4e5b-ad37-a9adfa98cd96 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.155706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.156354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.156641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.156980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.167344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.168892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:08.186109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:08.188467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:08.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:08.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:08.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:08.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:08.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 607/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:08.193563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.298765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199460-111e-42cc-9c86-ef72d1c431f8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.308307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:08.333879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.334528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.334816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.335095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.335790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:08.336099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.337469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.350299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 631/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:08.356093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.462357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c63c0-3513-4adc-a836-81f6b432ed6e None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.496253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.496894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.497179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.497458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.500706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:08.505783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.507277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:08.519198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:08.520385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:08.521331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 608/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:08.526334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.627666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e8a2e-fe79-47d8-9f88-bed755b99b41 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.663343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.663966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.664253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.664533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.665340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:08.665654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.667124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.684632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 632/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:45:08.691392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.797049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c901f-c81a-4ece-aef2-607559481a6f None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.832616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.833307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.833591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.833874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.898385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:08.904756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.906213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.008878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 609/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:45:09.015079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.043640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:09.117883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585043b7-1763-4357-a160-c3c0015348ad None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.154088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.155221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.155508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.155786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.165977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.167429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:09.182553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:09.184493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:09.185849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:09.185849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:09.185849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:09.185849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:09.185849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 633/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:09.190186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.286234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:09.293064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732144e7-7341-4335-8c78-b2434fded87c None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.327497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.328113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.328398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.328675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.329362 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:09.329665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.331048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.344532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 610/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:09.350461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.431262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:09.455906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2736-22e7-4d5b-a2f1-6c9371c80038 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.490922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.491565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.491851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.492237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.499590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.501050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:09.512356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:09.519327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:09.520429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 634/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:09.521022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.629921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4c5e6-846a-41ed-90ff-f0ba19a132bc None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.665247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.665872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.666157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.666463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.667216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:09.667529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.669118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a0d0-1dd3-4154-91f5-5f2fbc719a60 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.682831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 611/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:09.689338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.795664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83822b-4842-4dc4-99cb-4d74d527a874 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.832953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.833644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.833930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.834246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.835033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] RBAC: Adding query filter params (role.id=cf2a61f9748b49fbac555810a3da89da, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=90c8ff3469974d15a2a3ec8d7af9fe07) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:09.835342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.836784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d03570-0deb-404b-9f5f-289888cc8d53 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.843432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 635/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/role_assignments?user.id=90c8ff3469974d15a2a3ec8d7af9fe07&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=cf2a61f9748b49fbac555810a3da89da => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:11.953357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d92c-ea83-432a-8885-78b79a7bd4d9 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:11.953915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d92c-ea83-432a-8885-78b79a7bd4d9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:11.953915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d92c-ea83-432a-8885-78b79a7bd4d9 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:11.955338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 612/1247] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:11.960586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:11.961038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:11.961129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:11.967719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:11.980025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:11.981310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.050818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 636/1248] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:12.074311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012d92c-ea83-432a-8885-78b79a7bd4d9 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.094666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.178829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012d92c-ea83-432a-8885-78b79a7bd4d9 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.214222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.214889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.215178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.215459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.225790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:45:12.230978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:12.232116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:12.241349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 613/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:12.242165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.338395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501f53c-ddad-4935-85b8-bca1fb5f4f8f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.372628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.373326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.373995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.374280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.374588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.375310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:12.375614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.376990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.384926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 637/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:12.391356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.469588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.493491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457d39e-a5bd-4a60-a4c6-089be31251c1 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.528404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.529163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.529449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.529728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.539879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.541323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:12.556906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:12.558049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:12.558990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:12.558990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:12.558990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:12.558990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:12.558990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 614/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:12.565357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.666681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371be68c-0f1d-4a15-86ed-6258bcbc0bc8 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.702470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.703090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.703369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.703710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.704413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:12.704727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.706092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.718113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 638/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:12.724159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.824917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0acfd3-3a18-497d-aa97-0cf8e152e365 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.854672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.863554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.864198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.864488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.864768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.872177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.874417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:12.885733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:12.886963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:12.888183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 615/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:12.892959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.954915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.999196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6e41-a501-4eef-a19d-fd29eee89ee6 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.032803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.033422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.033706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.033985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.034810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:13.035118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.043873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.054304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 639/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:13.060114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.163276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307323e6-03c8-4a02-90df-eca7b6edf9fd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.196883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.197495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.197781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.198098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.252865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:13.265054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.266543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.365535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 616/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:45:13.371565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.471521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf449a9f-202b-4494-b00a-8499d8962d99 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.484613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:13.509481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.510488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.510977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.511410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.522379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.523873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:13.539743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:13.540867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:13.541993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:13.541993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:13.541993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:13.541993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:13.541993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 640/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:13.546818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.631616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:13.643821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966442-fec0-4694-bca8-bfec99330b28 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.677768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.678411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.678696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.678970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.679765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:13.680075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.681520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.693282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 617/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:13.699045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.798771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b87e-e95f-4a92-bce8-59ff23170e47 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.833433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.834075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.834372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.834655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.841459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.848056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:13.854527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:13.855697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:13.856822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 641/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:13.861986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.967699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ff8c-bda7-4b71-aeb5-ea0660a3adeb None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.002472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.003181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.003469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.003746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.004887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:14.005208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.006821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f1143-2a03-4ee7-a282-75c38fcb9ead None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.019255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 618/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:14.025540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.051844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:14.126289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57d640-4549-43c6-b8c9-992cf3777e72 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.158576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.159188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.159473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.159753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.160584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=90c8ff3469974d15a2a3ec8d7af9fe07) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:14.160989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.163362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5ab8-92bf-4465-bc89-1e5c81540755 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.170532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 642/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:45:14 2022] GET /identity/v3/role_assignments?user.id=90c8ff3469974d15a2a3ec8d7af9fe07&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:16.175646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7dce-a8d2-4ea7-a9a1-c0a81e3476f0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.176151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7dce-a8d2-4ea7-a9a1-c0a81e3476f0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.176151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7dce-a8d2-4ea7-a9a1-c0a81e3476f0 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.177489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 619/1261] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:16.182902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.183321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.183470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.202700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:16.203958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:16.271735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 643/1262] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:16.294142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7dce-a8d2-4ea7-a9a1-c0a81e3476f0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.307319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.402276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7dce-a8d2-4ea7-a9a1-c0a81e3476f0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.440903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.441496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.441786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.442073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.452312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:45:16.457289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:16.458430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:16.459300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 620/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:16.464536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.484092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.565333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bdbe0-9aeb-4f2e-9168-4001766a70f5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.600998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.601613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.601898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.602173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.602925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:16.603243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.604611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.613730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 644/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:16.619592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.689369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.722851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce03f7-6d04-446e-b797-1b95a0c79330 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.756418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.757060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.757369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.757650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.767709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.769211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:16.784358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:16.785279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:16.786503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:16.786503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:16.786503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:16.786503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:16.786503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 621/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:16.791768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.882025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.904275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0607-d554-4d46-9388-adc5699a1131 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.938028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.938665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.938947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.939224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.939912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:16.940219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.941588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.953534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 645/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:16.959038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.063088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50916f12-2e1b-4289-b3c0-3c67423181ff None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.082912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:17.097162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.097800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.098090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.098391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.105830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.107475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:17.118506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:17.119643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:17.120603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 622/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:17.125787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.227213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364d5d-0c01-4662-ae46-22863c1e5847 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.260820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.261523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.261814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.262099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.262935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:17.264157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.265354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.274068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:17.283227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 646/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:17.289511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.393307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b67877-35b9-41fa-ba32-09cc04f70140 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.427967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.428602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.428888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.429166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.440259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.441702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:17.457487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:17.458673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:17.459776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:17.459776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:17.459776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:17.459776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:17.459776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 623/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:17.464620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.563820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca0b7c-c3ae-4cb2-9270-7651aaab5732 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.599705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.600490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.600763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.601047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.601742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:17.602053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.603472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.615166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 647/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:17.621124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.655955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:17.722090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04084b1-b48e-4c87-b7d2-23cc3b71087b None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.757178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.757827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.758114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.758418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.765776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.767259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:17.778321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:17.779340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:17.780227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 624/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:17.785396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:17.849319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:17.892749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f8c8-1c33-412e-8ce2-1479f95cd496 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:17.927973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:17.928533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:17.928821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:17.930826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:17.931585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:17.931900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:17.933359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fed6b1-c183-4225-81d0-d623fc3fb540 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:17.946306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 648/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:45:17.952479 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:18.048118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:18.056639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde873f-765d-424e-ab38-6ccc5be20ca0 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:18.090410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:18.091107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:18.091387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:18.091667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] PATH_INFO: `/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/90c8ff3469974d15a2a3ec8d7af9fe07/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:18.093591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] RBAC: Authorizing `identity:create_grant(project_id=9b133df81b374a9e944d96078dfd1d55, user_id=90c8ff3469974d15a2a3ec8d7af9fe07, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:18.095097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dc270-36f3-4607-81d5-525af575e44b None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:18.285659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 625/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Thu Jul 28 17:45:17 2022] PUT /identity/v3/projects/9b133df81b374a9e944d96078dfd1d55/users/90c8ff3469974d15a2a3ec8d7af9fe07/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:20.282663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f370-7c5c-431e-bddd-43af957230e2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.283192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f370-7c5c-431e-bddd-43af957230e2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.283283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f370-7c5c-431e-bddd-43af957230e2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.284574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 649/1274] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:20.289914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.290314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.290518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.308648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:20.310035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:20.378367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 626/1275] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:20.401685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5f370-7c5c-431e-bddd-43af957230e2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.506174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5f370-7c5c-431e-bddd-43af957230e2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.524265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:20.540686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.541259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.541541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.541816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.552495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:45:20.557368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:20.558493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:20.560247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 650/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:20.565475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.651325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:20.669429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3171ccc-0a51-4db2-873a-ba837e34bfc6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.708500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.709118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.709402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.709680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.710416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:20.710725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.712086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.720547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 627/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:20.726567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.832065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa67de-1d1a-4054-adbc-f0130b7df581 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.866434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.867078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.867347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.867621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.878999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.880471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:20.896129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:20.897258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:20.898449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:20.898449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:20.898449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:20.898449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:20.898449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 651/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:20.903295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.008031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eabad3-fda9-41f3-80be-ff131fd692bd None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.041873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.042519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.042808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.043159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.043895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:21.044736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.046109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.058146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 628/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:21.064729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.099719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.171735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19e52-c4b6-4f32-afc5-88084d750c34 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.205425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.206059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.206370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.206654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] PATH_INFO: `/v3/users/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.214828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.216798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:45:21.227896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:21.228944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:21.229895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 652/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:21.235788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.239664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.337120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c377-8d31-4c00-833f-5e81c882f1ff None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.376695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.377237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.377526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.377808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.378648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:21.378959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.380493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.398853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 629/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:21.405295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.503579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.513207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33998e8-feed-421d-8d51-d1e8c447724b None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.548414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.549067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.549384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.549666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.617688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.619183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.708720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.719887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 653/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:45:21.725729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.787123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.824339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58aabb-1cd6-4e8e-9d18-26bd12dda977 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.860500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.861137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.861421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.861704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.871745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.873324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:45:21.887594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:21.889399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:21.895066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:45:21.895066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:45:21.895066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:45:21.895066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:21.895066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 630/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:21.896060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.999965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140837fb-c83f-48e1-93b1-10447baf6efd None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:22.034436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:22.035042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:22.035325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:22.035601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:22.036558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:22.036860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:22.038219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:22.049594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 654/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:22.055486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:22.158241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50be3-1f41-4891-969f-bd30b8d053bf None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:22.172328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:22.192578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:22.193250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:22.193538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:22.193813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:22.201011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:22.202581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:45:22.214994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:22.216162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: ERROR keystone.server.flask.application Jul 28 17:45:22.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 631/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Thu Jul 28 17:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:22.222274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:22.274922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:22.325827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0820d-95dc-441c-ba6b-84da3391c6e3 None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:22.359527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:22.360159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:22.360626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:22.361420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:22.362174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:22.362539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:22.363974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b1fb4-f5c5-4b07-988b-d2197c9dcab8 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:22.375999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 655/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Thu Jul 28 17:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:22.382127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:22.482471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19529-08ae-4882-a756-c846b736b320 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:22.516124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:22.516734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:22.517018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:22.517294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:22.518140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] RBAC: Adding query filter params (role.id=8fa73b7838ff4d4588a6fe9867cae064, scope.project.id=9b133df81b374a9e944d96078dfd1d55, user.id=90c8ff3469974d15a2a3ec8d7af9fe07) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:22.518484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:22.519904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda8eb6-56af-4f33-b823-f7c75e8d08ef None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:22.525857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 632/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Thu Jul 28 17:45:22 2022] GET /identity/v3/role_assignments?user.id=90c8ff3469974d15a2a3ec8d7af9fe07&scope.project.id=9b133df81b374a9e944d96078dfd1d55&role.id=8fa73b7838ff4d4588a6fe9867cae064 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:24.598598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9db8-cb82-4358-a383-58d4ad02e117 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.599140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9db8-cb82-4358-a383-58d4ad02e117 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.599140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9db8-cb82-4358-a383-58d4ad02e117 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.600581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 656/1288] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:24.605935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.606954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.607309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.626179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:24.627409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:24.657317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:24.698106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 633/1289] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:24.721118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9db8-cb82-4358-a383-58d4ad02e117 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.825881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9db8-cb82-4358-a383-58d4ad02e117 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.838053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:24.861369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.861940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.862220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.862529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] PATH_INFO: `/v3/services/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.863151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.864512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:24.873702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:24.875850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:24.877040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 657/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:24.881904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.988779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d367edd-303d-4828-bb97-ca5cc49ea331 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:25.024612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:25.025230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:25.025517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:25.025796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:25.026513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:25.026871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:25.028859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f155-4cf1-4741-8258-56c40ef07036 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:25.033781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 634/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:25.040179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:25.146732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3604d-22ef-435f-93af-e40da0a6c14a None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:25.184169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:25.184851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:25.185213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:25.185401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] PATH_INFO: `/v3/services` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:25.186082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] RBAC: Adding query filter params (type=placement) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:25.186407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:25.187768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6d46c-37e8-4b31-81f1-8f05a5e7b7ed None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:25.192181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 658/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Thu Jul 28 17:45:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:27.227675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352223-22d8-480d-9172-6fc7fc934a80 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:27.228316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352223-22d8-480d-9172-6fc7fc934a80 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:27.228316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352223-22d8-480d-9172-6fc7fc934a80 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:27.229546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 635/1293] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:27.235053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32caac-9f6f-4e8e-8ca0-e594cde8f0cd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:27.235494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32caac-9f6f-4e8e-8ca0-e594cde8f0cd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:27.235630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32caac-9f6f-4e8e-8ca0-e594cde8f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:27.254551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9b32caac-9f6f-4e8e-8ca0-e594cde8f0cd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:27.255787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9b32caac-9f6f-4e8e-8ca0-e594cde8f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:27.288279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:27.327064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 659/1294] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:27.351476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352223-22d8-480d-9172-6fc7fc934a80 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:27.383658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:27.456234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352223-22d8-480d-9172-6fc7fc934a80 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:27.500434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:27.501061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:27.501344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:27.501620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:27.502313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:27.503725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf711f-180d-4cb8-87d7-3ef71c85b3ed None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:27.609471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 636/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Thu Jul 28 17:45:27 2022] POST /identity/v3/services => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:29.670491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b863dd-5dc8-431a-b3f0-6507794accba None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:29.671067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b863dd-5dc8-431a-b3f0-6507794accba None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:29.671067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b863dd-5dc8-431a-b3f0-6507794accba None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:29.672422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 660/1296] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:29.677843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:29.678288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:29.678472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:29.697610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:29.698832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:29.781423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 637/1297] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:29.805849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b863dd-5dc8-431a-b3f0-6507794accba None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:29.921958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b863dd-5dc8-431a-b3f0-6507794accba None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:29.960593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:29.966176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:29.966176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:29.966176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] PATH_INFO: `/v3/services/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:29.966176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:29.966176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:29.974150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:29.981070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:29.982265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 661/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:45:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:29.984937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:30.005209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:30.098129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c174e-2497-4cc1-b831-8ee0d96a2333 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:30.134398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:30.135288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.135578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.135863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.136554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:30.136865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:30.139064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:30.144737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 638/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Thu Jul 28 17:45:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:30.150892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:30.185315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:30.258588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729b279-1f88-4f68-97f8-199c32a464be None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:30.292259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:30.292873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.293156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.293435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.295127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] RBAC: Adding query filter params (interface=public, service_id=a23b61a540de41048b66e85882e44eb6, region_id=RegionOne) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:30.297287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:30.297287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a136f2-7ba1-4ac5-b938-b1ecd3b9bd90 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:30.301733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 662/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Thu Jul 28 17:45:30 2022] GET /identity/v3/endpoints?service_id=a23b61a540de41048b66e85882e44eb6&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:30.307470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:30.388803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:30.407184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3a0ca-f3cb-409c-87a1-7d84a35649c6 None admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:30.441846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:30.442394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.442684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.442964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.443691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:30.445477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76705b69-c4f0-4d76-b5c5-0a6f774d8332 None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:30.453868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 639/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Thu Jul 28 17:45:30 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:32.450546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc33d-51ff-480b-a0b3-1d7756580c31 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.451310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc33d-51ff-480b-a0b3-1d7756580c31 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.451310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc33d-51ff-480b-a0b3-1d7756580c31 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.453107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 663/1302] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:32.458241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.458697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.458840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.477448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:32.478740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:32.546158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 640/1303] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:45:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:45:32.568758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc33d-51ff-480b-a0b3-1d7756580c31 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.665591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:32.673101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc33d-51ff-480b-a0b3-1d7756580c31 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:32.708479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:32.709040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.709322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.709600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] PATH_INFO: `/v3/services/placement` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.710235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:32.711654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:45:32.720597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:45:32.721776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application with Lock( Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: ERROR keystone.server.flask.application Jul 28 17:45:32.722711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 664/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Thu Jul 28 17:45:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:45:32.728052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.806530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:32.830261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92881815-5055-49f4-9a96-6038b22ac515 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:32.864747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:32.865358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.865638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.865915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] PATH_INFO: `/v3/services` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.866612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] RBAC: Adding query filter params (name=placement) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:32.866916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:32.868274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7536eb-f903-4dde-ba32-9b5e74dcbabd None admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:32.874048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 641/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Thu Jul 28 17:45:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:32.879997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.982528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39d604-c146-4454-81ae-7c3539638ecc None admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.019882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.020902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.021634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.021915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.022683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.024121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09096870-5839-4662-ab2c-ae4f3289b409 None admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.132938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 665/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:42.921727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=30 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:43.128370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:04.026533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0ae1-4a13-4fb1-be05-962d97263eff None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:04.026942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0ae1-4a13-4fb1-be05-962d97263eff None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:04.027032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0ae1-4a13-4fb1-be05-962d97263eff None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:04.028381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 642/1307] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:04.033810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0a56-bcea-42eb-b285-27a45841eab1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:04.034247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0a56-bcea-42eb-b285-27a45841eab1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:04.034596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0a56-bcea-42eb-b285-27a45841eab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:04.112768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4d3c0a56-bcea-42eb-b285-27a45841eab1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:04.115307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d3c0a56-bcea-42eb-b285-27a45841eab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:04.264179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 666/1308] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:04.317702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:04.317993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:04.318513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:04.319880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 643/1309] 10.222.0.85 () {54 vars in 965 bytes} [Thu Jul 28 17:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:04.326537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39180c60-3e5a-46be-88b4-48e78cb6ebc9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:04.326968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39180c60-3e5a-46be-88b4-48e78cb6ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:04.327328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39180c60-3e5a-46be-88b4-48e78cb6ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:04.415518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-39180c60-3e5a-46be-88b4-48e78cb6ebc9 None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:04.416691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-39180c60-3e5a-46be-88b4-48e78cb6ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:04.438965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:04.508166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 667/1310] 10.222.0.85 () {58 vars in 1062 bytes} [Thu Jul 28 17:53:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:04.518010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:04.519875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:04.640786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe46841-3c15-43c8-bc24-a2febc3628f5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:04.729788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:04.730336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:04.730707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:04.731036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:04.732694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:04.792903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:04.864637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:04.867611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bc9d2-2ae1-48c8-9688-c995a027e0f8 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:04.891041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 644/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:07.205961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c88a-8549-466b-97e1-c4a9e9614692 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.206575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c88a-8549-466b-97e1-c4a9e9614692 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.206575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c88a-8549-466b-97e1-c4a9e9614692 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.207871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 668/1312] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:07.213215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77888-2d8f-4a3c-bce0-56ac6809002f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.213692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77888-2d8f-4a3c-bce0-56ac6809002f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.213804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77888-2d8f-4a3c-bce0-56ac6809002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.308988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-68a77888-2d8f-4a3c-bce0-56ac6809002f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:07.310296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-68a77888-2d8f-4a3c-bce0-56ac6809002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:07.417629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 645/1313] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:07.470582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.470884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.471241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.473295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 669/1314] 10.222.0.85 () {54 vars in 965 bytes} [Thu Jul 28 17:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:07.483998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bc61-098d-4715-9422-4ff6134a19b1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.484295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bc61-098d-4715-9422-4ff6134a19b1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.484580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255bc61-098d-4715-9422-4ff6134a19b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.498155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:07.581478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f255bc61-098d-4715-9422-4ff6134a19b1 None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:07.583394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f255bc61-098d-4715-9422-4ff6134a19b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:07.678163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 646/1315] 10.222.0.85 () {58 vars in 1062 bytes} [Thu Jul 28 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:07.687629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:07.689314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:07.805566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de7db0-cce8-4c27-8094-8ac25df35c82 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.885335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:07.887940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.888556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.888839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.889114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.890762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:08.001815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:08.003396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c82758-59d3-4971-bf7a-3e18a75ec7ae service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:08.022163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 670/1316] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:10.969221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.969795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.969795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.971176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 647/1317] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:10.977632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2651-4124-4b81-82ec-8655225bfcc7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.978092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2651-4124-4b81-82ec-8655225bfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.978289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2651-4124-4b81-82ec-8655225bfcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.048227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-024e2651-4124-4b81-82ec-8655225bfcc7 None None] MFA Rules not processed for user `59c9455e874f42308d26ae107456be33`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:11.049454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-024e2651-4124-4b81-82ec-8655225bfcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:11.108404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.155432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 671/1318] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:10 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:11.196327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.197954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:11.316811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ff51b-8366-4a6b-8c1c-eaf4d1a2e6b5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.319772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.403343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.404217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.404343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.404534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.406164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:11.524307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.525760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea2365-ee81-4041-8af4-211699d14055 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.544764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 648/1319] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:14.451909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.452472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.452472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.453802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 672/1320] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:14.459606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12a33f-b26c-4a57-988d-ae45baab6594 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.460035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12a33f-b26c-4a57-988d-ae45baab6594 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.460173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12a33f-b26c-4a57-988d-ae45baab6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.500698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.525767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3e12a33f-b26c-4a57-988d-ae45baab6594 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:14.527024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3e12a33f-b26c-4a57-988d-ae45baab6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:14.622569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 649/1321] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:14.662889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.664428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:14.771162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860f5b3-06c9-432c-be5e-78f7ad7ce453 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.844789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.845412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.845724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.845999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.847659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:14.856326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.961883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.963362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416b40b-d67f-49a4-8f6e-1ed7093d705e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.981443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 673/1322] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:18.315416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.315952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.315952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.318077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 650/1323] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:18.323426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce333-0768-46a3-a168-8041f269efcf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.323871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce333-0768-46a3-a168-8041f269efcf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.323966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce333-0768-46a3-a168-8041f269efcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.390711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-34bce333-0768-46a3-a168-8041f269efcf None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:18.392173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-34bce333-0768-46a3-a168-8041f269efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:18.450177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.490028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 674/1324] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:18.528224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.529768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:18.655322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7dc85-72d2-4ba8-8a07-7aeaf83d2f18 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.671786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.751369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.752003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.752291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.752571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.754698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:18.885926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.887415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97643f91-69d2-493b-a15c-5db033492d9e service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.911544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 651/1325] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:20.991355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.991927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.991927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.993273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 675/1326] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:20.998838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1c81-1398-4ccc-ba62-677aa63549a4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.999315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1c81-1398-4ccc-ba62-677aa63549a4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.999522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1c81-1398-4ccc-ba62-677aa63549a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.048998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.067694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9c1e1c81-1398-4ccc-ba62-677aa63549a4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:21.068983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9c1e1c81-1398-4ccc-ba62-677aa63549a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:21.172030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 652/1327] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:21.209230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.210821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:21.314177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ec50a-db21-43c6-9ec4-e2122a573f46 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.386813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.387711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.387994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.388268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.389872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:21.408051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.500463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.501914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f92ce-f53b-453d-8329-608862f0e166 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.521511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 676/1328] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:23.607544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.608209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.608209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.609780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 653/1329] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:23.617940 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9b22-7e78-4c03-9228-818fc2f5718e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.618562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9b22-7e78-4c03-9228-818fc2f5718e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.618562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9b22-7e78-4c03-9228-818fc2f5718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.697256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-892a9b22-7e78-4c03-9228-818fc2f5718e None None] MFA Rules not processed for user `59c9455e874f42308d26ae107456be33`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:23.698673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-892a9b22-7e78-4c03-9228-818fc2f5718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.772779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.810296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 677/1330] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:23.849237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.850849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.965530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90a12d-8885-4c33-add0-cd374a43e1f8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.989895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.055579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.056213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.056499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.056780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.058495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.180291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.181745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b75e5f-62f9-477e-852d-a07b966ce552 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.201477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 654/1331] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:26.975457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.976018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.976018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.977406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 678/1332] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:26.982621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e7be-8f20-43ac-9278-ffb78936d9b1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.983017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e7be-8f20-43ac-9278-ffb78936d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.983196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e7be-8f20-43ac-9278-ffb78936d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.050134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.053857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7e89e7be-8f20-43ac-9278-ffb78936d9b1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:27.055175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7e89e7be-8f20-43ac-9278-ffb78936d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.155488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 655/1333] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:27.192637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.194186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.305169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89a42-7c73-4b70-9e38-ca46395088ee None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.380503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.381119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.381405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.381684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.383356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.410687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.493172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.494744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bfdf9-59cc-41dd-a5c3-905863f72889 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.513081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 679/1334] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:29.586119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.586689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.586816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.588115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 656/1335] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:29.593206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74a5a-54eb-4b55-b8d3-9d37c2888c2d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.593621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74a5a-54eb-4b55-b8d3-9d37c2888c2d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.593798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e74a5a-54eb-4b55-b8d3-9d37c2888c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.665022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-58e74a5a-54eb-4b55-b8d3-9d37c2888c2d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:29.666541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-58e74a5a-54eb-4b55-b8d3-9d37c2888c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:29.742635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.764532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 680/1336] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:29.806311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.808784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:29.914939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fed22-c1f9-4006-ac45-ed2541c679f3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.943803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.994144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.994797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.995079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.995354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.996981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:30.107444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.110097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50f2a-24fb-47d0-9817-ed0aa29d55cc service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.127707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 657/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:32.989226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:32.989790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:32.989790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:32.991266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 681/1338] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:32.996814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb50c3d-0efc-4065-93e5-44d7ed47ce90 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:32.997233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb50c3d-0efc-4065-93e5-44d7ed47ce90 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:32.997381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb50c3d-0efc-4065-93e5-44d7ed47ce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.066207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6eb50c3d-0efc-4065-93e5-44d7ed47ce90 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:33.067468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6eb50c3d-0efc-4065-93e5-44d7ed47ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:33.075837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.164272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 658/1339] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:33.200374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.202023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:33.306852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f912a25-59f6-4a73-a30c-a99a63bbcea9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.380748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.381361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.381644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.381920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.383661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:33.420373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.497797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.499290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aba9ba-ae25-44a9-bd1f-b68f096e9806 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.519955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 682/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:37.985541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.986071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.986171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.988436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 659/1341] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:37.994341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690e74f-89dd-4f42-bd96-cd13315680f8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.006752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690e74f-89dd-4f42-bd96-cd13315680f8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.006752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690e74f-89dd-4f42-bd96-cd13315680f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.084036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6690e74f-89dd-4f42-bd96-cd13315680f8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:38.085312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6690e74f-89dd-4f42-bd96-cd13315680f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:38.190716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.217910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 683/1342] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:38.257747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.259650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:38.374697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c279-765b-4f99-962b-a7276afd8e6a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.414959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.456361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.457102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.457374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.457655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.459342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:38.571708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.573071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abd414-2d99-4664-90fc-fb073b5f709e service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.592500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 660/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:41.793347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.794149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.794149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.795451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 684/1344] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:41.801471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132d80-a35f-4cb5-9ed8-548920372d7b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.801973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132d80-a35f-4cb5-9ed8-548920372d7b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.802070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed132d80-a35f-4cb5-9ed8-548920372d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.871849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-ed132d80-a35f-4cb5-9ed8-548920372d7b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:41.873076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ed132d80-a35f-4cb5-9ed8-548920372d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:41.895615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.987673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 661/1345] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:42.025495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.027238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:42.133274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e05b8-5f2c-405f-8077-16af4cce9a4e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.208724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.209347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.209632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.209913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.211577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:42.258388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.329808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.331870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567979d4-08aa-4363-936a-860f524df8f4 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.351695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 685/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:46.805386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.805981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.806090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.807548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 662/1347] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:46.813208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ec68-3c71-4257-b98d-f491461d154f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.821336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ec68-3c71-4257-b98d-f491461d154f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.821336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ec68-3c71-4257-b98d-f491461d154f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.888012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3bf8ec68-3c71-4257-b98d-f491461d154f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:46.889260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3bf8ec68-3c71-4257-b98d-f491461d154f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:46.997680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.000309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 686/1348] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:47.041210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.043022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:47.164502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bff64e-f8a6-4e8f-bc79-802f966da922 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.217762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.245339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.245958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.246239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.246540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.248155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:47.363755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.365188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0218908-81a2-46fc-9698-e0d0883564e2 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.384715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 663/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:52.496165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be758-4d38-4b53-972e-a3038b40cfa2 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.496956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be758-4d38-4b53-972e-a3038b40cfa2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.497752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be758-4d38-4b53-972e-a3038b40cfa2 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.499339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 687/1350] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 17:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:52.506651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9659-bf71-4e9b-9436-d1b7ed127f95 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.507654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9659-bf71-4e9b-9436-d1b7ed127f95 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.508706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb9659-bf71-4e9b-9436-d1b7ed127f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.667211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c8fb9659-bf71-4e9b-9436-d1b7ed127f95 None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:52.668670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c8fb9659-bf71-4e9b-9436-d1b7ed127f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:52.689392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.795757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 664/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 17:53:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:56.911756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017de7-591f-4040-98ac-bd174347109d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.912435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017de7-591f-4040-98ac-bd174347109d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.912778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017de7-591f-4040-98ac-bd174347109d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.915728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 688/1352] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 17:53:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:56.921538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ced2a-0384-4467-a57c-fdf4ad9c0733 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.922001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ced2a-0384-4467-a57c-fdf4ad9c0733 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.922337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ced2a-0384-4467-a57c-fdf4ad9c0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.005985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-995ced2a-0384-4467-a57c-fdf4ad9c0733 None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:57.007548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-995ced2a-0384-4467-a57c-fdf4ad9c0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.104249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.145703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 665/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:58.653380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba866-ca66-49aa-aebb-0a5dfd0beb8e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.653954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba866-ca66-49aa-aebb-0a5dfd0beb8e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.653954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ba866-ca66-49aa-aebb-0a5dfd0beb8e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.655338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 689/1354] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 17:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:58.661401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60b0e-984d-45ec-831b-cdba81c9846f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.666273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60b0e-984d-45ec-831b-cdba81c9846f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.666273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60b0e-984d-45ec-831b-cdba81c9846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.746367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-02c60b0e-984d-45ec-831b-cdba81c9846f None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.747590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-02c60b0e-984d-45ec-831b-cdba81c9846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.864954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 666/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:02.328934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9717f-2a8a-41d4-8e9b-51454c43604f None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.329580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9717f-2a8a-41d4-8e9b-51454c43604f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.329876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9717f-2a8a-41d4-8e9b-51454c43604f None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.332174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 690/1356] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:02.337914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995ef6-d6d1-4a23-8976-6ce8097a5582 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.338459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995ef6-d6d1-4a23-8976-6ce8097a5582 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.338553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995ef6-d6d1-4a23-8976-6ce8097a5582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.400643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:02.445981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-af995ef6-d6d1-4a23-8976-6ce8097a5582 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:02.448866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-af995ef6-d6d1-4a23-8976-6ce8097a5582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:02.571381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 667/1357] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:02.627118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.634224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.634224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.634224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 691/1358] 10.222.0.85 () {54 vars in 962 bytes} [Thu Jul 28 17:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:02.638779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1ded-33d1-4e02-a0ce-1d564c18af7f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.639445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1ded-33d1-4e02-a0ce-1d564c18af7f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.640243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1ded-33d1-4e02-a0ce-1d564c18af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.733243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fa1a1ded-33d1-4e02-a0ce-1d564c18af7f None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:02.734482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fa1a1ded-33d1-4e02-a0ce-1d564c18af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:02.792010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:02.875089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 668/1359] 10.222.0.85 () {58 vars in 1059 bytes} [Thu Jul 28 17:54:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:02.883425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:02.886988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:03.048163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858c45-6d6d-4393-af5d-209198ceb639 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.129632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:03.173086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.173717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.174000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.174274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.176465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:03.280466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.282081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49781e-8237-448f-92a8-97918d9672cc service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.302690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 692/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:05.598712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.599230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.599320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.600865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 669/1361] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:05.606573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb1c3-ba98-41df-84a9-28b0282c80ad None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.607472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb1c3-ba98-41df-84a9-28b0282c80ad None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.608039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb1c3-ba98-41df-84a9-28b0282c80ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.705384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-748cb1c3-ba98-41df-84a9-28b0282c80ad None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:05.707529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-748cb1c3-ba98-41df-84a9-28b0282c80ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:05.753957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:05.820924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 693/1362] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:05.873146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.877099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:06.082341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e07136-114c-4d1d-ad81-e9afd2e7b8d4 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:06.092633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.245117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:06.245735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.246015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.246290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.247995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:06.344696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:06.346393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922fb57a-0e62-4180-a4cb-01d82c4665b5 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:06.366884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 670/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:08.876915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227afcd1-b3cb-49cc-b5a7-0f7f374c7861 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.877511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227afcd1-b3cb-49cc-b5a7-0f7f374c7861 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.877615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227afcd1-b3cb-49cc-b5a7-0f7f374c7861 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.879378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 694/1364] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:08.885418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2e309-b229-4e06-ac23-57a11424e7c6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.885955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2e309-b229-4e06-ac23-57a11424e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.886247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2e309-b229-4e06-ac23-57a11424e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.906428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:08.975645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-70d2e309-b229-4e06-ac23-57a11424e7c6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:08.975645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-70d2e309-b229-4e06-ac23-57a11424e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.085101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 671/1365] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:09.135708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.136384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.136726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.138293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 695/1366] 10.222.0.85 () {54 vars in 962 bytes} [Thu Jul 28 17:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:09.145180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ef3d-b708-442b-8e6e-819b76df1d07 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.146640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ef3d-b708-442b-8e6e-819b76df1d07 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.146640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ef3d-b708-442b-8e6e-819b76df1d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.228541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6e80ef3d-b708-442b-8e6e-819b76df1d07 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:09.229767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6e80ef3d-b708-442b-8e6e-819b76df1d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.266578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.353735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 672/1367] 10.222.0.85 () {58 vars in 1059 bytes} [Thu Jul 28 17:54:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:09.361880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.363519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.529109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899d75b-cde6-4082-880d-61079c34a3dd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.551455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.655630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.656669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.657183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.657680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.660204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.765690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.767342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53c03-0eb7-4591-aac9-44e9b18b9a15 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.787357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 696/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:54:10.766911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225687-faf8-4da5-9504-593921403c88 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.767482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225687-faf8-4da5-9504-593921403c88 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.767482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225687-faf8-4da5-9504-593921403c88 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.769556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 673/1369] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:10.774909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ef8a-f1ee-4c81-bfdd-b31f974775ca None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.775382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ef8a-f1ee-4c81-bfdd-b31f974775ca None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.775481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ef8a-f1ee-4c81-bfdd-b31f974775ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.809680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.852268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-eb29ef8a-f1ee-4c81-bfdd-b31f974775ca None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:10.853496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-eb29ef8a-f1ee-4c81-bfdd-b31f974775ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.957848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 697/1370] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:11.302550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:11.302550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:11.302550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:11.305085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 674/1371] 10.222.0.85 () {54 vars in 964 bytes} [Thu Jul 28 17:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:11.308737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade353-c39c-422a-ad25-1abd6685dc31 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:11.309031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade353-c39c-422a-ad25-1abd6685dc31 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:11.309316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade353-c39c-422a-ad25-1abd6685dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:11.394025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-94ade353-c39c-422a-ad25-1abd6685dc31 None None] MFA Rules not processed for user `4573990adc20477ab369cd717aaf0679`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:11.395603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-94ade353-c39c-422a-ad25-1abd6685dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:11.447207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:11.477715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 698/1372] 10.222.0.85 () {58 vars in 1061 bytes} [Thu Jul 28 17:54:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:11.485576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:11.487781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:11.601725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91703e40-f08a-433f-9a76-e78c69df6f84 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:11.649178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:11.687326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:11.687951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:11.688233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:11.688509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:11.690527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:11.810529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:11.812016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14cd0-785d-42b3-b927-281028c16a4c service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:11.833085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 675/1373] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:54:12.117716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.118148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.118442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.119833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 699/1374] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:12.125374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d4229-8f74-43dc-aa03-ef51f1bd14bf None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.125646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d4229-8f74-43dc-aa03-ef51f1bd14bf None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.125923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d4229-8f74-43dc-aa03-ef51f1bd14bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.196465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-628d4229-8f74-43dc-aa03-ef51f1bd14bf None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:12.197883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-628d4229-8f74-43dc-aa03-ef51f1bd14bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:12.219631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.304501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 676/1375] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:12.349626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:12.351170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:12.504196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cbc58-b18d-43b7-aff2-a42cf2765f58 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:12.558591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.625310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:12.625932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.626220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.626526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.628379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:12.749127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:12.750836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f70240-e7c4-4991-a3ee-800b420c5c74 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:12.786473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 700/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:15.070086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.070633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.070726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.072105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 677/1377] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:15.077283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341fde-d56d-42b5-bed7-27f6f691fe05 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.077663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341fde-d56d-42b5-bed7-27f6f691fe05 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.077854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341fde-d56d-42b5-bed7-27f6f691fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.149394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bf341fde-d56d-42b5-bed7-27f6f691fe05 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:15.150650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bf341fde-d56d-42b5-bed7-27f6f691fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:15.157954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.283321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 701/1378] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:15.327199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.328725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:15.476481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f73f9-0633-45a5-956a-218e6e331716 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.480156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.604812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.605441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.605722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.605998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.608128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:15.725437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.727871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b780ea-6546-454f-95af-27bf0d1bba11 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:15.751539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 678/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:17.926136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.926687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.926807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.928227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 702/1380] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:17.933289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc27c-8cd8-41b3-8603-256300fd112d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.933696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc27c-8cd8-41b3-8603-256300fd112d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.933881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc27c-8cd8-41b3-8603-256300fd112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.945768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.003531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0f8cc27c-8cd8-41b3-8603-256300fd112d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:18.005933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0f8cc27c-8cd8-41b3-8603-256300fd112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:18.115473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 679/1381] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:18.160959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.162624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:18.309557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.320162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561da672-617a-4689-af53-b54ae5f91aa0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.441606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.442439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.442720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.442995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.444615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:18.548018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.550651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fc13d-2006-4311-b396-8a84886004e1 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.562393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.571395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 703/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:21.082717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.083361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.083361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.084647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 680/1383] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:21.090038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7594-b934-4d53-8afd-78aa5ffcbda7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.090513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7594-b934-4d53-8afd-78aa5ffcbda7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.090606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee7594-b934-4d53-8afd-78aa5ffcbda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.167677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bdee7594-b934-4d53-8afd-78aa5ffcbda7 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:21.168842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bdee7594-b934-4d53-8afd-78aa5ffcbda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:21.269673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 704/1384] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:21.313239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:21.314875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:21.410439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:21.475321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71465d-6ca9-425a-93df-b9563fda7cda None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:21.597014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:21.597648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.597937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.598222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.599915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:21.652987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:21.695194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:21.696740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1a62-e5bb-4215-80ee-bb27655c6506 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:21.716080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 681/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:23.800507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:23.801205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:23.801205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:23.802587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 705/1386] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:23.808155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa7c3-47f5-42f9-871c-e4cd3412cd70 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:23.808693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa7c3-47f5-42f9-871c-e4cd3412cd70 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:23.808820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa7c3-47f5-42f9-871c-e4cd3412cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:23.880106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-34dfa7c3-47f5-42f9-871c-e4cd3412cd70 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:23.881182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-34dfa7c3-47f5-42f9-871c-e4cd3412cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:23.995554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 682/1387] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:24.041121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:24.042781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:24.121837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:24.196657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fb6d9-d6c8-48f3-be42-93a8b508dab3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:24.336224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:24.336863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:24.337171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:24.337453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:24.339253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:24.388232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:24.450012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:24.451627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5435f4-612b-48c0-b750-02d332051394 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:24.471028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 706/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:26.696736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:26.697278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:26.697278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:26.698701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 683/1389] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:26.703969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387c9af-293b-4e60-89d0-1adc76009cc6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:26.704398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387c9af-293b-4e60-89d0-1adc76009cc6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:26.705207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387c9af-293b-4e60-89d0-1adc76009cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:26.771356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4387c9af-293b-4e60-89d0-1adc76009cc6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:26.772552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4387c9af-293b-4e60-89d0-1adc76009cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:26.874369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 707/1390] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:26.918900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:26.920451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.072306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.191348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8161f-7c81-4010-b889-74a558b5b52f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:27.295172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.307058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:27.311190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.311190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.311190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.311190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.404949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:27.406521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c342a-084c-4eda-b6ff-0e780d08b5b8 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:27.426065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 684/1391] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:27.491887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e4dd9-528d-4f37-871f-da6bddc5208e None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.492181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e4dd9-528d-4f37-871f-da6bddc5208e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.492486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e4dd9-528d-4f37-871f-da6bddc5208e None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.493865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 708/1392] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:27.498986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8f0f-0eec-4a81-b88a-4f79de2dfebe None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.499280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8f0f-0eec-4a81-b88a-4f79de2dfebe None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.499563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8f0f-0eec-4a81-b88a-4f79de2dfebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.569784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cafa8f0f-0eec-4a81-b88a-4f79de2dfebe None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:27.571046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cafa8f0f-0eec-4a81-b88a-4f79de2dfebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.636837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.683654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 685/1393] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:27.753080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.753667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.753807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.756151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 709/1394] 10.222.0.85 () {54 vars in 966 bytes} [Thu Jul 28 17:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:27.765154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7b98e-d5a9-4d75-a776-0e6a4068ae44 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.765857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7b98e-d5a9-4d75-a776-0e6a4068ae44 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.765857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7b98e-d5a9-4d75-a776-0e6a4068ae44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.871994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b4e7b98e-d5a9-4d75-a776-0e6a4068ae44 None None] MFA Rules not processed for user `9e8966ba17c04e1782a6ed9fb4e59254`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:27.873229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b4e7b98e-d5a9-4d75-a776-0e6a4068ae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.963490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 686/1395] 10.222.0.85 () {58 vars in 1063 bytes} [Thu Jul 28 17:54:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:27.971622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.973205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.988465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:28.088607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cecde-dc0f-4ed4-88b2-c12aceef0b68 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:28.166721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:28.167353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:28.167641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:28.167919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:28.169577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:28.240547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:28.281881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:28.283908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d4a6d-5cc6-4822-ad2f-3b9c57f681c2 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:28.304737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 710/1396] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:29.413108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4936aa-fabc-41f5-82b1-70b5e14ac670 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:29.413665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4936aa-fabc-41f5-82b1-70b5e14ac670 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:29.413665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4936aa-fabc-41f5-82b1-70b5e14ac670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:29.485258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0a4936aa-fabc-41f5-82b1-70b5e14ac670 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:29.487114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0a4936aa-fabc-41f5-82b1-70b5e14ac670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:29.496035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:29.576259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 687/1397] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 17:54:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:29.623885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797882d3-8f7d-4975-a953-443495e9f77c None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:29.624390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797882d3-8f7d-4975-a953-443495e9f77c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:29.624497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797882d3-8f7d-4975-a953-443495e9f77c None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:29.625832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 711/1398] 10.222.0.85 () {54 vars in 943 bytes} [Thu Jul 28 17:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:29.637600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:29.637600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:29.637600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:29.638323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387f0d-80d0-4099-bbcd-0851ed9b1469 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:29.638323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387f0d-80d0-4099-bbcd-0851ed9b1469 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:29.638323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80387f0d-80d0-4099-bbcd-0851ed9b1469 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:29.639023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 712/1399] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:29.644284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:29.644579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:29.645416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:29.714015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:29.715285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:29.723295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] MFA Rules not processed for user `8b5ce51b2ba94ce3ad65f1d61c3efe60`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:29.723295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:29.807840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:29.819267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 713/1400] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:29.830301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 688/1401] 10.222.0.85 () {58 vars in 1040 bytes} [Thu Jul 28 17:54:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:29.840602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:29.841837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:29.864621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:29.865827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:29.871491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:29.949197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6613-b6a3-4497-a2f3-916eeccdf9d4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:29.966043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590eb9-daa7-410c-9130-bed596e0ee2a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.058406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.059368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.059654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.059931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.061883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.083448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.090569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.091199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.091481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.091761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.093538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.108592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.157440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.159738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59badd-6c73-401e-a78f-9c91b1c10d03 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.168651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 714/1402] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 17:54:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:54:30.207110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.208709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d65407-49dc-4ebc-bb29-a0e3482a912e service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.228708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 689/1403] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:54:32.586150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:32.586837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:32.586837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:32.588120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 715/1404] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:32.593281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a072cd-18cb-4585-a3f0-76552536531c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:32.593742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a072cd-18cb-4585-a3f0-76552536531c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:32.593905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a072cd-18cb-4585-a3f0-76552536531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:32.671651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f3a072cd-18cb-4585-a3f0-76552536531c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:32.673988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f3a072cd-18cb-4585-a3f0-76552536531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:32.747473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:32.817330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 690/1405] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:32.850409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:32.851948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:33.044391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5eb2f2-e5b1-4f83-b7d7-bdc14ef1eb9a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.096674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:33.182956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.183586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.183867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.184149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.185798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:33.286518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.288145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25ddfc-3039-49f9-8c42-b6286a3c2514 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.310043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 716/1406] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:35.604471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.605040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.605040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.606607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 691/1407] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:35.612903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357cc5-43d0-4a3b-a634-1a726291aaca None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.613444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357cc5-43d0-4a3b-a634-1a726291aaca None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.613575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357cc5-43d0-4a3b-a634-1a726291aaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.665962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:35.687764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6c357cc5-43d0-4a3b-a634-1a726291aaca None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:35.689357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6c357cc5-43d0-4a3b-a634-1a726291aaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:35.802468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 717/1408] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:35.848590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.852010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:36.115798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bc986-5aa6-4153-a642-c0e1a62de8e0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:36.196633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:36.273588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:36.274739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:36.274739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:36.274739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:36.279313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:36.396344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:36.398382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c62079-2fb5-4b0c-8261-8d755cf2bb20 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:36.421249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 692/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:45.800014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=47 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:46.418598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=75 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:54.349905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746d67-d3a2-4f68-947d-48a23f6e8769 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:54.350535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746d67-d3a2-4f68-947d-48a23f6e8769 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:54.350535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746d67-d3a2-4f68-947d-48a23f6e8769 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:54.352031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 718/1410] 10.222.0.85 () {54 vars in 918 bytes} [Thu Jul 28 17:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:54.357275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6335a-9d8c-4552-858c-133699e697bc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:54.357744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6335a-9d8c-4552-858c-133699e697bc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:54.357852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6335a-9d8c-4552-858c-133699e697bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:54.440441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3aa6335a-9d8c-4552-858c-133699e697bc None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:54.441746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3aa6335a-9d8c-4552-858c-133699e697bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:54.558824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 693/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Thu Jul 28 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:54.575128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365b97-e2d5-4ccb-9e1f-d6dc8f919b85 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:54.575438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365b97-e2d5-4ccb-9e1f-d6dc8f919b85 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:54.575720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365b97-e2d5-4ccb-9e1f-d6dc8f919b85 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:54.577107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 719/1412] 10.222.0.85 () {54 vars in 918 bytes} [Thu Jul 28 17:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:54.582199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e89c-b37e-4b01-ab57-50775e803e81 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:54.582622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e89c-b37e-4b01-ab57-50775e803e81 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:54.582909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e89c-b37e-4b01-ab57-50775e803e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:54.661079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:54.663930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-97c0e89c-b37e-4b01-ab57-50775e803e81 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:54.665171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-97c0e89c-b37e-4b01-ab57-50775e803e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:54.794367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 694/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Thu Jul 28 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:57.288352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.288996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.288996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.290368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 720/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:54:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:57.295888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651de8-7473-4f7e-9cf2-80f5393926b6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.297079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651de8-7473-4f7e-9cf2-80f5393926b6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.297079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651de8-7473-4f7e-9cf2-80f5393926b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.366705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f4651de8-7473-4f7e-9cf2-80f5393926b6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:57.367935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f4651de8-7473-4f7e-9cf2-80f5393926b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:57.426415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:57.469856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 695/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:57.514614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:57.516360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:57.675155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54a0a8-cb40-4dd1-9963-8923719a48b8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:57.752610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:57.794162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:57.794863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.795136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.795420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.797747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:57.892574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:57.894217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766c506-a46a-4378-b8fa-655a815229a0 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:57.914032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 721/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:58.839220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:58.839718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:58.839821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:58.841205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 696/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:58.847274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ea9-f1db-4b7a-82d7-ed9a355056e0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:58.847750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ea9-f1db-4b7a-82d7-ed9a355056e0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:58.847884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ea9-f1db-4b7a-82d7-ed9a355056e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:58.929453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c4614ea9-f1db-4b7a-82d7-ed9a355056e0 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:58.930715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c4614ea9-f1db-4b7a-82d7-ed9a355056e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:58.958723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:59.029572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 722/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:59.181409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:59.182886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:59.332400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c5748-a656-4c59-a981-250b7f44f49c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:59.373990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:59.450501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:59.451033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:59.451309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:59.451579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:59.453396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:59.548373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:59.550121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef36c-2799-4aeb-af60-6330368ffcae service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:59.570017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 697/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:00.469613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.470208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.470208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.471597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 723/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:00.476676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8b81-0389-46b2-acd7-29646ea6e3b8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.477124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8b81-0389-46b2-acd7-29646ea6e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.477265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8b81-0389-46b2-acd7-29646ea6e3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.533902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:00.557383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-832d8b81-0389-46b2-acd7-29646ea6e3b8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:00.559039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-832d8b81-0389-46b2-acd7-29646ea6e3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:00.659216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 698/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:00.813277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:00.814933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:00.968010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52763f66-21af-4db2-a140-ef32869d5f26 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:00.981135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:01.084305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:01.084849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:01.085132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:01.085408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:01.087075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:01.180390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:01.182035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b62151-15d3-4e61-8b3f-50343e95a862 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:01.201833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 724/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:02.073663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:02.074252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:02.074252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:02.075608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 699/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:02.080979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0a3b-c1c7-4ac9-935a-4c1054ba4d7e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:02.081448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0a3b-c1c7-4ac9-935a-4c1054ba4d7e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:02.081575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c0a3b-c1c7-4ac9-935a-4c1054ba4d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:02.101911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:02.161009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-338c0a3b-c1c7-4ac9-935a-4c1054ba4d7e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:02.162289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-338c0a3b-c1c7-4ac9-935a-4c1054ba4d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:02.258735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 725/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:02.410839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:02.412273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:02.524434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:02.565536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fc5db-286e-4259-9040-dc06c41db385 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:02.685104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:02.685633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:02.685912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:02.686186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:02.688387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:02.768973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:02.782933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:02.784615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50a929-ad9e-49de-bc4a-d7bb6766faae service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:02.802954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 700/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:03.684885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.685469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.685469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.686981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 726/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:03.692017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88f8ed-6180-4946-a33a-4702ca0b5023 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.692482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88f8ed-6180-4946-a33a-4702ca0b5023 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.692588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88f8ed-6180-4946-a33a-4702ca0b5023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.769510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3e88f8ed-6180-4946-a33a-4702ca0b5023 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:03.770680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3e88f8ed-6180-4946-a33a-4702ca0b5023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:03.869312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 701/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:04.022633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:04.024140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:04.112052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:04.175663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61079b7-55e5-4eb2-ad68-40f7c7c65638 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:04.294253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:04.294845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:04.295094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:04.295370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:04.297254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:04.348065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:04.394739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:04.396554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4a162-07d0-4059-9c06-8c46f562bf70 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:04.417080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 727/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:05.325073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:05.325627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:05.325627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:05.327114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 702/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:05.332895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a86d-9a9c-46f5-b5cf-357d144fae67 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:05.333343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a86d-9a9c-46f5-b5cf-357d144fae67 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:05.333450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a86d-9a9c-46f5-b5cf-357d144fae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:05.413781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bf72a86d-9a9c-46f5-b5cf-357d144fae67 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:05.415030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bf72a86d-9a9c-46f5-b5cf-357d144fae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:05.513160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 728/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:05.667171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:05.668800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:05.720725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:05.823004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2594e5a-66b3-446f-bbf2-98b6fba308e9 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:05.944008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:05.944508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:05.944794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:05.945072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:05.946747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:05.954424 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:06.050617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:06.052307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c3e81-76f6-436c-a608-bb6f8df94a7e service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:06.071762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 703/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:06.996463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.997048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.997048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.998437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 729/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:07.003853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc043-b1d8-4376-9579-5d8bf48c659e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:07.004345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc043-b1d8-4376-9579-5d8bf48c659e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:07.004454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc043-b1d8-4376-9579-5d8bf48c659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:07.091032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f2ebc043-b1d8-4376-9579-5d8bf48c659e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:07.092164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f2ebc043-b1d8-4376-9579-5d8bf48c659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:07.169544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:07.201751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 704/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:07.355943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:07.357450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:07.376522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:07.516356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39be5e-53e1-47ff-80fa-b7c1a53f4021 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:07.609617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:07.641269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:07.642305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:07.642847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:07.643334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:07.646415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:07.742857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:07.744581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f06c1c-f04d-42d1-b680-42b8efd69d1f service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:07.764235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 730/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:08.679865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:08.680446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:08.680446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:08.681827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 705/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:08.687632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1383d4-01fb-495a-9878-9a4b2c8a9f7c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:08.688072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1383d4-01fb-495a-9878-9a4b2c8a9f7c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:08.688308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1383d4-01fb-495a-9878-9a4b2c8a9f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:08.767642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7a1383d4-01fb-495a-9878-9a4b2c8a9f7c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:08.768886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7a1383d4-01fb-495a-9878-9a4b2c8a9f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:08.809713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:08.872803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 731/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:09.026009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:09.027569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:09.176550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5269-1ff5-4f7f-9c6a-85e4fba69eb9 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:09.232293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:09.293289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:09.293917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:09.294202 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:09.294510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:09.296645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:09.389042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:09.391799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81a809-880e-417b-8836-7b8b3397e834 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:09.412378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 706/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:10.294207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:10.294817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:10.294817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:10.296313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 732/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:10.301382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d985e2-95ec-4762-822d-7fc1ca8d4976 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:10.301849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d985e2-95ec-4762-822d-7fc1ca8d4976 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:10.302015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d985e2-95ec-4762-822d-7fc1ca8d4976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:10.380405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-19d985e2-95ec-4762-822d-7fc1ca8d4976 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:10.381632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-19d985e2-95ec-4762-822d-7fc1ca8d4976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:10.383770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:10.483568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 707/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:10.638961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:10.641150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:10.789744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff3b9d-62ed-4787-8c69-c94f512547ad None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:10.816583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:10.911224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:10.911760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:10.912051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:10.912331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:10.913970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:11.004462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:11.006090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b1dff-de9f-4fb1-86da-4ae96e67c58f service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:11.025052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 733/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:12.956838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:12.957374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:12.957374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:12.958793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 708/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:12.964120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ccd-469f-40b5-9f28-283c3bb5db3b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:12.964533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ccd-469f-40b5-9f28-283c3bb5db3b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:12.964688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ccd-469f-40b5-9f28-283c3bb5db3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:13.009178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:13.038917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-38675ccd-469f-40b5-9f28-283c3bb5db3b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:13.040159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-38675ccd-469f-40b5-9f28-283c3bb5db3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:13.139308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 734/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:13.178278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:13.179828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:13.287581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ac687-ad5e-498b-aaef-63b6bab71d8c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:13.320700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:13.364092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:13.364716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:13.365001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:13.365280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:13.366964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:13.477734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:13.479138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8f157-feaa-40a0-b0b0-9c33c821d3c8 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:13.497748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 709/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:15.720689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:15.721259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:15.721375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:15.722839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 735/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:15.728519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb18e8-d113-41c6-bb9a-83c3d976eb22 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:15.728956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb18e8-d113-41c6-bb9a-83c3d976eb22 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:15.729057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb18e8-d113-41c6-bb9a-83c3d976eb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:15.798416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-00bb18e8-d113-41c6-bb9a-83c3d976eb22 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:15.799669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00bb18e8-d113-41c6-bb9a-83c3d976eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:15.813229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:15.900003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 710/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:15.938659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:15.940263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:16.050946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64edc8d-e067-4874-b4ef-0854da1b5ac8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:16.063997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:16.129281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:16.129907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:16.130209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:16.130498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:16.132168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:16.253109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:16.254585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf49565-7c20-4dd7-af5e-08fd4ab981ef service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:16.273360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 736/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:18.717095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:18.717776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:18.717776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:18.719300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 711/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:18.724662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bd0ea-ddba-4541-a786-ec17d186e880 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:18.725086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bd0ea-ddba-4541-a786-ec17d186e880 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:18.725207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bd0ea-ddba-4541-a786-ec17d186e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:18.775448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:18.792933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ad8bd0ea-ddba-4541-a786-ec17d186e880 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:18.794181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ad8bd0ea-ddba-4541-a786-ec17d186e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:18.894333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 737/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:18.935037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:18.936690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:19.043826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0508b-3ba4-4b6e-8604-42ab4d2f2ec1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:19.087458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:19.122075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:19.122739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:19.123070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:19.123442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:19.125720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:19.239708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:19.241211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39e179-aba7-44a0-b705-153c59ee00bd service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:19.260351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 712/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:21.432995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:21.433592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:21.433592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:21.434952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 738/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:21.440478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6fd65-ee04-477f-b718-cb58e2315386 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:21.440961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6fd65-ee04-477f-b718-cb58e2315386 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:21.441098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6fd65-ee04-477f-b718-cb58e2315386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:21.508719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-98f6fd65-ee04-477f-b718-cb58e2315386 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:21.509930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-98f6fd65-ee04-477f-b718-cb58e2315386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:21.532682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:21.613341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 713/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:21.653549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:21.655774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:21.763702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc09af-d07f-4536-b44a-8acd12761188 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:21.785571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:21.846123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:21.846783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:21.847075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:21.847357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:21.848997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:21.959895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:21.961341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714346a-928f-4b52-a9e0-1f778c48bf0e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:21.979313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 739/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:24.122356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:24.122929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:24.122929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:24.124473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 714/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:24.129981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2588885-5fb1-4546-9880-ccb211a11007 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:24.130433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2588885-5fb1-4546-9880-ccb211a11007 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:24.130557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2588885-5fb1-4546-9880-ccb211a11007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:24.195543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:24.198378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f2588885-5fb1-4546-9880-ccb211a11007 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:24.199609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f2588885-5fb1-4546-9880-ccb211a11007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:24.298110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 740/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:24.338409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:24.339952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:24.456251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb64b5-175a-4a57-90ca-9f34351ae9f6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:24.506479 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:24.533351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:24.533975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:24.534771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:24.534771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:24.536341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:24.647835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:24.649777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58c579-f0ae-45c0-bc9c-a9da99ecd39c service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:24.668640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 715/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:26.793205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:26.793790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:26.793790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:26.795719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 741/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:55:26.801526 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed58e-96b1-4ce6-a6fe-4e3b874814f5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:26.802073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed58e-96b1-4ce6-a6fe-4e3b874814f5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:26.802157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed58e-96b1-4ce6-a6fe-4e3b874814f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:26.872720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8c9ed58e-96b1-4ce6-a6fe-4e3b874814f5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:26.874580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8c9ed58e-96b1-4ce6-a6fe-4e3b874814f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:26.905145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:26.978011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 716/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:27.018091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:27.019678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:27.138743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50953-28bc-4dd4-ab5e-a1a367d9c546 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:27.168607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:27.217211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:27.218101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:27.218529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:27.218812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:27.220451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:27.341078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:27.343191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a85a9-74ab-4693-acc3-bfd17154634d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:27.363118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 742/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:29.466604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:29.467409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:29.467409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:29.468592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 717/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:29.474185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfdac1-6424-4741-99ac-2c9dc1ba507b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:29.474740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfdac1-6424-4741-99ac-2c9dc1ba507b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:29.474857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfdac1-6424-4741-99ac-2c9dc1ba507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:29.558665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5edfdac1-6424-4741-99ac-2c9dc1ba507b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:29.561132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5edfdac1-6424-4741-99ac-2c9dc1ba507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:29.575329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:29.679224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 743/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:29.719174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:29.720758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:29.833121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c349d-4b7f-46fe-8c77-30143dfbc352 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:29.891715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:29.911061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:29.911687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:29.911981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:29.912257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:29.913912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:30.031425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:30.032875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dffb4e-cf3f-4acd-8b5c-50e43a50d913 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:30.053006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 718/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:32.144909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:32.145437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:32.145437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:32.146840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 744/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:32.152226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284900-9a36-4ea8-a5e9-6962a9818b59 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:32.155888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284900-9a36-4ea8-a5e9-6962a9818b59 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:32.155888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284900-9a36-4ea8-a5e9-6962a9818b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:32.219718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f9284900-9a36-4ea8-a5e9-6962a9818b59 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:32.220937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f9284900-9a36-4ea8-a5e9-6962a9818b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:32.260182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:32.319938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 719/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:32.358938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:32.360527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:32.465787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dee9b8-e8ef-4bea-b16e-1c948a43e983 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:32.626826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:32.663737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:32.664365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:32.664672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:32.664953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:32.667164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:32.784979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:32.786469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640848f-7914-4266-878d-edbadf5e735c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:32.806704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 745/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:55:34.853505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:34.853969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:34.854079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:34.855486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 720/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:34.861124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327e82-f595-47a4-a0cb-74bbaeadcf29 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:34.861628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327e82-f595-47a4-a0cb-74bbaeadcf29 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:34.861737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327e82-f595-47a4-a0cb-74bbaeadcf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:34.927604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-81327e82-f595-47a4-a0cb-74bbaeadcf29 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:34.928829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-81327e82-f595-47a4-a0cb-74bbaeadcf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:34.947721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:35.026150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 746/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:35.064657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:35.066194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:35.172076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764fd67-816e-4b59-9a13-e75c54a2122e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:35.243558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:35.251107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:35.255665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:35.255665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:35.255665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:35.255665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:35.366827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:35.368277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053391ef-41c2-43f4-b6b2-e8bedadffe57 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:35.387880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 721/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:37.458323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:37.458896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:37.458896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:37.460285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 747/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:37.465627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a86e1-d354-443b-913e-51e86cb0200a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:37.466054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a86e1-d354-443b-913e-51e86cb0200a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:37.466826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a86e1-d354-443b-913e-51e86cb0200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:37.536221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-332a86e1-d354-443b-913e-51e86cb0200a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:37.537446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-332a86e1-d354-443b-913e-51e86cb0200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:37.589416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:37.650739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 722/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:37.689551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:37.691124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:37.814403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03130ad-f47b-4570-bd5d-2ca6ee232f6d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:37.870144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:37.899905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:37.900525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:37.900805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:37.901077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:37.902772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:38.028959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:38.031127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734e8e-b468-47a8-842f-c06eec73171e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:38.051486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 748/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:47.648780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=22 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:48.049145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=66 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:57.918599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e69b-02a7-4e9f-b66f-829a6dcfbdd9 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:57.919210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e69b-02a7-4e9f-b66f-829a6dcfbdd9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:57.919210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e69b-02a7-4e9f-b66f-829a6dcfbdd9 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:57.920545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 723/1471] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:57.926073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2c5a-3f44-414c-a4d2-b0a9e04c9d81 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:57.926590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2c5a-3f44-414c-a4d2-b0a9e04c9d81 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:57.926725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2c5a-3f44-414c-a4d2-b0a9e04c9d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:58.005506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cf2b2c5a-3f44-414c-a4d2-b0a9e04c9d81 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:58.006744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cf2b2c5a-3f44-414c-a4d2-b0a9e04c9d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:58.112630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 749/1472] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:58.201790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:58.202073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:58.202368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:58.203731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 724/1473] 10.222.0.85 () {54 vars in 966 bytes} [Thu Jul 28 17:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:58.208800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924829-72e9-4feb-af43-1bf1383075d4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:58.209160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924829-72e9-4feb-af43-1bf1383075d4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:58.209446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924829-72e9-4feb-af43-1bf1383075d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:58.285844 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a6924829-72e9-4feb-af43-1bf1383075d4 None None] MFA Rules not processed for user `9e8966ba17c04e1782a6ed9fb4e59254`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:58.287080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a6924829-72e9-4feb-af43-1bf1383075d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:58.308018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:58.367455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 750/1474] 10.222.0.85 () {58 vars in 1063 bytes} [Thu Jul 28 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:55:58.375601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:58.377128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:58.489001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17d3c4-b821-41e7-b06d-432be233d92e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:58.572363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:58.572986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:58.573268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:58.573543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:58.575900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:58.630824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:58.686854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:58.688304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3ebf4-6b58-485e-8514-40b7fcfae550 service glance] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:58.707063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 725/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:00.825756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:00.826323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:00.826323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:00.827906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 751/1476] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:00.833433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563f17-1018-403a-9956-b5a13d081b5f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:00.833937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563f17-1018-403a-9956-b5a13d081b5f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:00.834040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563f17-1018-403a-9956-b5a13d081b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:00.912703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c2563f17-1018-403a-9956-b5a13d081b5f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:00.913970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c2563f17-1018-403a-9956-b5a13d081b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:01.035901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 726/1477] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:01.060665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:01.062264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:01.208941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d799f-cc25-4734-bf9a-8418c9c2c210 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:01.263554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:01.310596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:01.311217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:01.311502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:01.311783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:01.312468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:01.314201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:56:01.314201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 17:56:01.314201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 17:56:01.315400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e42e26-a21e-4117-94d4-167061cf2081 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:01.334915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 752/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:01 2022] GET /identity/v3/projects => generated 2553 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:03.425915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:03.426477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:03.426477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:03.427763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 727/1479] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:03.432935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2623a-0ffd-4bc9-94f2-9c452e14834f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:03.433378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2623a-0ffd-4bc9-94f2-9c452e14834f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:03.433521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2623a-0ffd-4bc9-94f2-9c452e14834f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:03.503343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a7a2623a-0ffd-4bc9-94f2-9c452e14834f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:03.504596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a7a2623a-0ffd-4bc9-94f2-9c452e14834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:03.613244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 753/1480] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:03.661310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:03.663906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:03.675166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:03.833187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec360-b3d2-4a05-bda0-131f32b26e1d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:03.913755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:03.946323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:03.946992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:03.947274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:03.947549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:03.950555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:04.040798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:04.042318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2192c80-0663-47ff-bbd9-c17acbb03527 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:04.061108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 728/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:06.262648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:06.263213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:06.263213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:06.264572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 754/1482] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:06.269872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a4e2-6fd7-46f7-8171-12dfd768ad68 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:06.270273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a4e2-6fd7-46f7-8171-12dfd768ad68 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:06.270472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a4e2-6fd7-46f7-8171-12dfd768ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:06.336194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4e53a4e2-6fd7-46f7-8171-12dfd768ad68 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:06.337497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4e53a4e2-6fd7-46f7-8171-12dfd768ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:06.387070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:06.440660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 729/1483] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:06.478854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:06.480586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:06.510934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:06.593505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2efdf-277f-4783-a940-5465d3ff2bbe None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:06.668756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:06.669374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:06.669663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:06.669943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:06.671609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:06.748571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:06.785701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:06.787334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89862cd-05e8-4a12-a060-ebf475bc13c0 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:06.805350 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 755/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:08.892950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:08.893530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:08.893530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:08.894915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 730/1485] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:08.900517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b85-2d87-4703-9d27-169ec702685b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:08.901013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b85-2d87-4703-9d27-169ec702685b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:08.901111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b85-2d87-4703-9d27-169ec702685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:08.967455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cf0f9b85-2d87-4703-9d27-169ec702685b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:08.968686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cf0f9b85-2d87-4703-9d27-169ec702685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:09.066575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 756/1486] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:56:09.108994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:09.110610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:09.225959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14532bf-12fa-477e-9eda-7e55d0611497 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:09.299234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:09.301052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:09.301673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:09.301956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:09.302237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:09.303922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:09.425633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:09.427036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f338d0-cfe9-47a3-a73e-d33442fe0bb6 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:09.445707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 731/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:11.653266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:11.653879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:11.653879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:11.655342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 757/1488] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:11.661882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b369a1-87c3-4dcf-a285-b1f45f0cad98 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:11.662333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b369a1-87c3-4dcf-a285-b1f45f0cad98 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:11.662564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b369a1-87c3-4dcf-a285-b1f45f0cad98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:11.735570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-84b369a1-87c3-4dcf-a285-b1f45f0cad98 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:11.736806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-84b369a1-87c3-4dcf-a285-b1f45f0cad98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:11.792033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:11.836662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 732/1489] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:11.881347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:11.884061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:11.927947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:11.999123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69264b3-d9da-4cb7-9bf7-a19efe71484a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:12.080870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:12.081502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:12.081789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:12.082068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:12.087682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:12.190679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:12.222808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:12.224250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4947c12-823f-4c93-93b9-be1820e2ecd6 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:12.243004 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 758/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:14.668533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:14.669100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:14.669100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:14.670375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 733/1491] 10.222.0.85 () {54 vars in 926 bytes} [Thu Jul 28 17:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:14.676274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d49f-7584-4fb2-a5a0-5947f63d0520 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:14.676945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d49f-7584-4fb2-a5a0-5947f63d0520 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:14.676945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3d49f-7584-4fb2-a5a0-5947f63d0520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:14.752123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-abf3d49f-7584-4fb2-a5a0-5947f63d0520 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:14.753356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-abf3d49f-7584-4fb2-a5a0-5947f63d0520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:14.855712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 759/1492] 10.222.0.85 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:14.897133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:14.899846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:15.009889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd70f0-acfe-4051-afdf-e9786d068212 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:15.091343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:15.091969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:15.092255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:15.092539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:15.094237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:15.103703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:15.219230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:15.220673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e6600-1cce-4bb8-b865-15b091ea6b96 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:15.249501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 734/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:24.853266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=82 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:25.247053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=50 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:05.833308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:05.833947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:05.833947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:05.919040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:58:05.920268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:06.026638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 760/1494] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:58:06.038848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.039525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.039525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.060119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:58:06.061342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:06.070082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 735/1495] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:58:06.075645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.093714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dffd6-801b-4814-b79c-81cc27f7e342 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.093714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.093714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.093714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 17:58:06.094285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 17:58:06.095051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.107434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 761/1496] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 17:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:58:06.116672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.126224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220889bd-116b-4f90-af50-147ed91612d6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.131216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.131825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.132134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.132409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.133077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.134956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.134956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 17:58:06.134956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 17:58:06.136318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.232904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 736/1497] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 17:58:06 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:58:06.237330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.247449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48676-6965-43b6-9e3e-090e846748f9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.297364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:06.304595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.305213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.305494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.305771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.306460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.307478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.307478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 17:58:06.307478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 17:58:06.308692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.387120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 762/1498] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 17:58:06 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:58:06.391237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.402484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac520c-9ea7-4995-a8dc-8f4c536ca1ac admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.407193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 17:58:06.427118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 17:58:06.427809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.431047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 737/1499] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:58:06.435253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.446059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31194b-719e-4612-b7e6-3b3803b886b1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.450739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.451351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.451632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.451911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.452600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.453610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.453610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 17:58:06.453610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 17:58:06.454546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdeb09b-075b-4412-aff4-665ff9d97298 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.469307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 763/1500] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:58:06.473686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.483892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc856-3d26-4484-b377-42d77415e444 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.488573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.489238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.489521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.489851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] PATH_INFO: `/v3/projects/50ecfee52beb4dfdbd4bd9252f07ac89/users/3ec8d439d9f545d6b204a82fba90c68b/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.505141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ecfee52beb4dfdbd4bd9252f07ac89, user_id=3ec8d439d9f545d6b204a82fba90c68b, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.506543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:06.506543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 17:58:06.506543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 17:58:06.508517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d483663-c999-4804-a0cf-60ea01e9bde3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.580919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 738/1501] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 17:58:06 2022] PUT /identity/v3/projects/50ecfee52beb4dfdbd4bd9252f07ac89/users/3ec8d439d9f545d6b204a82fba90c68b/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:58:06.594009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c0e2-69eb-4468-be3d-b687e316d623 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.594298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c0e2-69eb-4468-be3d-b687e316d623 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.594612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099c0e2-69eb-4468-be3d-b687e316d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.622443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f099c0e2-69eb-4468-be3d-b687e316d623 None None] MFA Rules not processed for user `3ec8d439d9f545d6b204a82fba90c68b`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:58:06.623668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f099c0e2-69eb-4468-be3d-b687e316d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:06.686666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 764/1502] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:58:06.700558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1014d-b860-43e0-9aa4-3a24aa7e2910 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.700852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1014d-b860-43e0-9aa4-3a24aa7e2910 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.701130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1014d-b860-43e0-9aa4-3a24aa7e2910 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.702507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 739/1503] 10.222.0.85 () {54 vars in 964 bytes} [Thu Jul 28 17:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:58:06.707222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68abca-01b7-4ab2-b6fe-41769b21fb9d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.707511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68abca-01b7-4ab2-b6fe-41769b21fb9d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.708455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68abca-01b7-4ab2-b6fe-41769b21fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.766177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1c68abca-01b7-4ab2-b6fe-41769b21fb9d None None] MFA Rules not processed for user `4573990adc20477ab369cd717aaf0679`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:58:06.767413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1c68abca-01b7-4ab2-b6fe-41769b21fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:06.845574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 765/1504] 10.222.0.85 () {58 vars in 1061 bytes} [Thu Jul 28 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:58:06.854468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1014d-b860-43e0-9aa4-3a24aa7e2910 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:06.866397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1014d-b860-43e0-9aa4-3a24aa7e2910 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:06.869913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:06.870558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.870849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.871131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.880662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:06.882254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:06.892219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 740/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:58:06.941863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc5c-cb0d-43f5-9003-20ff7e45a546 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:06.942198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc5c-cb0d-43f5-9003-20ff7e45a546 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:06.943100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc5c-cb0d-43f5-9003-20ff7e45a546 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:06.944323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 766/1506] 10.222.0.85 () {52 vars in 830 bytes} [Thu Jul 28 17:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:58:06.948224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:07.016147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ab202-17c7-4df9-a5fd-e1892c41fc8a service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:07.021091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:07.021821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:07.022122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:07.022382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] PATH_INFO: `/v3/users/3ec8d439d9f545d6b204a82fba90c68b` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:07.023617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec8d439d9f545d6b204a82fba90c68b)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:07.029169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:07.029169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 17:58:07.029169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 17:58:07.029169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:07.266172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec8d439d9f545d6b204a82fba90c68b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:07.266895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec8d439d9f545d6b204a82fba90c68b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:07.267518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec8d439d9f545d6b204a82fba90c68b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:07.287418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec8d439d9f545d6b204a82fba90c68b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:07.356669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec8d439d9f545d6b204a82fba90c68b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:07.377040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:07.388529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 741/1507] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 17:58:06 2022] DELETE /identity/v3/users/3ec8d439d9f545d6b204a82fba90c68b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:58:07.393033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:07.393320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:07.393607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:07.414153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:58:07.415403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:07.424985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:07.477110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 767/1508] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 17:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:58:07.486949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:07.488506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:07.578732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f68108-871e-457f-9b66-7294754884c0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:07.582399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:07.583686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:07.583996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:07.584331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:07.593975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:07.595471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a688eab-60b0-48e2-91ff-0e943dbc6576 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:07.606395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 742/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:58:08.068243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:08.070443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:08.083380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d03b2e-a320-4231-aa2f-0f2243393b5a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:08.088357 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:08.089014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:08.089318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:08.089599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] PATH_INFO: `/v3/projects/50ecfee52beb4dfdbd4bd9252f07ac89` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:08.090808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ecfee52beb4dfdbd4bd9252f07ac89)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:08.091927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:58:08.091927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 17:58:08.091927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 17:58:08.093076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:08.210517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ecfee52beb4dfdbd4bd9252f07ac89', 'request_id': 'req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:08.224969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ecfee52beb4dfdbd4bd9252f07ac89', 'request_id': 'req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:08.231503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ecfee52beb4dfdbd4bd9252f07ac89', 'request_id': 'req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:08.306212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] The token cache is being invalidate because project 50ecfee52beb4dfdbd4bd9252f07ac89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:58:08.306556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:08.307336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9c5951d9-0a0a-4790-9bfe-3c4538368cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:08.308894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 768/1510] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 17:58:08 2022] DELETE /identity/v3/projects/50ecfee52beb4dfdbd4bd9252f07ac89 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:58:17.601867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=38 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:18.302939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:59:50.012843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:50.013460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:50.013460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:50.014859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 743/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:59:50.020279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823638d-73a8-4b87-a2a5-bff9e565affb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:50.020739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823638d-73a8-4b87-a2a5-bff9e565affb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:50.020885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823638d-73a8-4b87-a2a5-bff9e565affb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:50.043677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f823638d-73a8-4b87-a2a5-bff9e565affb None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:59:50.045389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f823638d-73a8-4b87-a2a5-bff9e565affb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:50.116881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 769/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:59:50.278758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:59:50.282290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:50.393097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5856f5-4cb8-49b3-bfa3-f1e80032dae1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:59:50.398931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:59:50.399555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:50.399838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:50.400118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:50.408925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:59:50.410560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9373438-dee6-4224-913e-e06582ac14b0 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:59:50.420775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 744/1513] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:59:51.274862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7c43-75f4-4fc2-bb81-898b3afef66b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:51.275434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7c43-75f4-4fc2-bb81-898b3afef66b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:51.275552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7c43-75f4-4fc2-bb81-898b3afef66b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:51.277016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 770/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:59:51.284860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac70e8-fb12-4065-8cb6-6d342c18de3d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:51.285342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac70e8-fb12-4065-8cb6-6d342c18de3d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:51.285440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac70e8-fb12-4065-8cb6-6d342c18de3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:51.305959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d0ac70e8-fb12-4065-8cb6-6d342c18de3d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:59:51.307278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d0ac70e8-fb12-4065-8cb6-6d342c18de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:51.315805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 745/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:59:51.470646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d7c43-75f4-4fc2-bb81-898b3afef66b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:59:51.485414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d7c43-75f4-4fc2-bb81-898b3afef66b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:59:51.491684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:59:51.492308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:51.492594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:51.492879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:51.502447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:59:51.504187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30264629-2131-4bee-b050-592b0cf205e8 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:59:51.514100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 771/1516] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 17:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:59:51.843523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3796-5ea4-4255-aa01-404e083bd7e0 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:51.843821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3796-5ea4-4255-aa01-404e083bd7e0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:51.844107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd3796-5ea4-4255-aa01-404e083bd7e0 None None] PATH_INFO: `/v3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:51.846066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 746/1517] 10.222.0.99 () {54 vars in 925 bytes} [Thu Jul 28 17:59:51 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:59:51.852698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2e1db-3536-4d54-93b3-fb21f83d9a53 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:51.852988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2e1db-3536-4d54-93b3-fb21f83d9a53 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:51.853270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2e1db-3536-4d54-93b3-fb21f83d9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:51.908394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-85a2e1db-3536-4d54-93b3-fb21f83d9a53 None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:59:51.910030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-85a2e1db-3536-4d54-93b3-fb21f83d9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:51.986854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 772/1518] 10.222.0.99 () {58 vars in 1015 bytes} [Thu Jul 28 17:59:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:01.100078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db482-8cff-4dfd-a0fd-409ddc7e8d2b None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:01.100965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db482-8cff-4dfd-a0fd-409ddc7e8d2b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:01.101140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28db482-8cff-4dfd-a0fd-409ddc7e8d2b None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:01.102815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 747/1519] 10.222.0.85 () {54 vars in 961 bytes} [Thu Jul 28 18:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:01.108429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028a80-74fb-4eb9-b83f-c89ac7e6edbf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:01.109038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028a80-74fb-4eb9-b83f-c89ac7e6edbf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:01.109038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028a80-74fb-4eb9-b83f-c89ac7e6edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:01.134692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9c028a80-74fb-4eb9-b83f-c89ac7e6edbf None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:01.138541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9c028a80-74fb-4eb9-b83f-c89ac7e6edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:01.151678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 773/1520] 10.222.0.85 () {58 vars in 1058 bytes} [Thu Jul 28 18:00:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:01.160578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28db482-8cff-4dfd-a0fd-409ddc7e8d2b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:01.175207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28db482-8cff-4dfd-a0fd-409ddc7e8d2b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:01.180693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:01.181317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:01.181601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:01.181879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:01.191437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:01.193054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76808-1b44-46c5-858e-d2435526ded1 service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:01.204399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 748/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:02.799913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb3300-3f8a-4210-b38d-6f1392a3212b None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:02.800500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb3300-3f8a-4210-b38d-6f1392a3212b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:02.800500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb3300-3f8a-4210-b38d-6f1392a3212b None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:02.801882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 774/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:02.806982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ddbe-f909-4217-8105-b2e61bbe66d2 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:02.807487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ddbe-f909-4217-8105-b2e61bbe66d2 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:02.807599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ddbe-f909-4217-8105-b2e61bbe66d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:02.834859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c871ddbe-f909-4217-8105-b2e61bbe66d2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:02.836155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c871ddbe-f909-4217-8105-b2e61bbe66d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:02.845563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 749/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:03.001531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb3300-3f8a-4210-b38d-6f1392a3212b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:03.016084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb3300-3f8a-4210-b38d-6f1392a3212b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:03.022271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:03.022921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:03.023208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:03.023500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:03.032011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:03.034424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dee3f3-1c88-4f77-8f58-2209adda25a8 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:03.044075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 775/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:03.884687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318f96-0d8c-4cc5-af45-9b98a2905abd None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:03.885217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318f96-0d8c-4cc5-af45-9b98a2905abd None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:03.885323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318f96-0d8c-4cc5-af45-9b98a2905abd None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:03.886709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 750/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:03.891939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7364c-eba6-4749-a577-cc428094b7ff None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:03.892458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7364c-eba6-4749-a577-cc428094b7ff None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:03.892566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7364c-eba6-4749-a577-cc428094b7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:03.914193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fdc7364c-eba6-4749-a577-cc428094b7ff None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:03.915454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fdc7364c-eba6-4749-a577-cc428094b7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:03.923991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 776/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:04.080004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8318f96-0d8c-4cc5-af45-9b98a2905abd None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:04.094655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8318f96-0d8c-4cc5-af45-9b98a2905abd None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:04.100057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:04.100666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:04.100951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:04.101229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:04.109844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:04.111470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879181a0-2c06-47ca-a505-6d39f4d93e14 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:04.121429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 751/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:08.113083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:08.113660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:08.113660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:08.115274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 777/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:08.122818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c10b-e4e3-4914-ad67-5e38c0ce48ac None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:08.123728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c10b-e4e3-4914-ad67-5e38c0ce48ac None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:08.123897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c10b-e4e3-4914-ad67-5e38c0ce48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:08.152524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4ac2c10b-e4e3-4914-ad67-5e38c0ce48ac None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:08.153995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4ac2c10b-e4e3-4914-ad67-5e38c0ce48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:08.164368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 752/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:08.289688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:08.291540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:08.329903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:00:08.366200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508ca93-f0ed-4bce-8334-f32e12d5fddd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:08.371208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:08.371838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:08.372128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:08.372413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:08.384002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:08.385456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e3bef-6587-43ef-a912-d0bdba447795 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:08.397116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 778/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:10.461545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:10.462128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:10.462128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:10.463508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 753/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:10.468821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a653-5688-422d-a613-3b0a8afa5d33 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:10.469897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a653-5688-422d-a613-3b0a8afa5d33 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:10.469994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a653-5688-422d-a613-3b0a8afa5d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:10.490658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f8b7a653-5688-422d-a613-3b0a8afa5d33 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:10.491874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f8b7a653-5688-422d-a613-3b0a8afa5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:10.499747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 779/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:00:10.537475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:10.539276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:10.553154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeef8e-fd3d-4e32-9d14-021de5d4ec0b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:10.556786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:10.557399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:10.557689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:10.557970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:10.567291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:10.568730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72a732-b172-4d5b-ad23-a9b4904c7001 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:10.578512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 754/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:14.113180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:14.113707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:14.113707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:14.115064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 780/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:14.120316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f343b7-3fd9-49ec-92c8-234c8767a9c2 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:14.120744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f343b7-3fd9-49ec-92c8-234c8767a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:14.120877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f343b7-3fd9-49ec-92c8-234c8767a9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:14.140261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-49f343b7-3fd9-49ec-92c8-234c8767a9c2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:14.142374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-49f343b7-3fd9-49ec-92c8-234c8767a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:14.150151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 755/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:00:14.457734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:14.459509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:14.517668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c0e58-f36d-4965-bc97-2e619bdb1380 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:14.521325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:14.521941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:14.522228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:14.522541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:14.531997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:14.533444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b90069-f37a-481c-badf-baa4852bd727 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:14.542941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 781/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:16.582873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:16.583445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:16.583545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:16.584902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 756/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:00:16.590280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eddda-1dc1-4792-90b7-09342473ddef None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:16.590748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eddda-1dc1-4792-90b7-09342473ddef None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:16.590871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eddda-1dc1-4792-90b7-09342473ddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:16.611679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-066eddda-1dc1-4792-90b7-09342473ddef None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:00:16.612890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-066eddda-1dc1-4792-90b7-09342473ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:16.620759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 782/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:00:16.930840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:16.932959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:16.947082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1dff4-2c90-4739-9e8d-55e4e1e83134 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:16.950870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:16.951506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:16.951787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:16.952066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:16.961921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:16.963390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d439874-5718-421a-84b4-177c854c2123 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:16.972889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 757/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:26.605857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=47 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:00:26.969311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=82 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:46.149787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674130-b8f0-4335-ba0e-99471992f835 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.153794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674130-b8f0-4335-ba0e-99471992f835 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.153794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9674130-b8f0-4335-ba0e-99471992f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.180822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d9674130-b8f0-4335-ba0e-99471992f835 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.181070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe97dd-8f04-4925-9751-fbc1ad69f5d9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.181549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe97dd-8f04-4925-9751-fbc1ad69f5d9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.181549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe97dd-8f04-4925-9751-fbc1ad69f5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.182052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d9674130-b8f0-4335-ba0e-99471992f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.202404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 783/1540] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.211668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.211973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.212251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.215880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fcfe97dd-8f04-4925-9751-fbc1ad69f5d9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.217365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fcfe97dd-8f04-4925-9751-fbc1ad69f5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.237195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 758/1541] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.247529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755a10-616d-45fe-b35b-59fd2ebec847 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.248175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755a10-616d-45fe-b35b-59fd2ebec847 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.248681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755a10-616d-45fe-b35b-59fd2ebec847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.253779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.253779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.265914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 784/1542] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:01:46.279425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.291666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-11755a10-616d-45fe-b35b-59fd2ebec847 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.293178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ae967-54ad-49d2-aae4-17f1e0a72ef3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.294538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-11755a10-616d-45fe-b35b-59fd2ebec847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.301867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.302711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.303002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.303287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.304000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:46.304313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.305975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.306209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 759/1543] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:01:46.307663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2f39-5807-40cf-9e5f-f9107e116bef None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.308890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2f39-5807-40cf-9e5f-f9107e116bef None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.309414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2f39-5807-40cf-9e5f-f9107e116bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.333404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 785/1544] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:01:46.334287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.338129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a7ae2f39-5807-40cf-9e5f-f9107e116bef None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.340631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a7ae2f39-5807-40cf-9e5f-f9107e116bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.356268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329b44-20b0-44ed-9e29-b457dbee58a5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.365928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 760/1545] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.367248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dade-37ad-4467-9328-323b205b726a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.367544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dade-37ad-4467-9328-323b205b726a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.367829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6dade-37ad-4467-9328-323b205b726a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.373139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.373893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.374090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.374517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.375233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:46.375546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.377158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.399316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 786/1546] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:46.400217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.420593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4ab6dade-37ad-4467-9328-323b205b726a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.423741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4ab6dade-37ad-4467-9328-323b205b726a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.424441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b622322-f853-4208-9990-ebe3420e6cb2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.433572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 761/1547] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.434719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.435515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.435867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.436161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.436448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.436688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.437131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.440910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:01:46.440910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 18:01:46.440910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 18:01:46.440910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f42c-4109-49f8-9e30-378e36209761 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.441729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.463105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.464923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.473036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 762/1548] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.473801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.483258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c01db0-e5f1-41e1-8fed-a75d10b67848 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.487830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.488418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.488740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.489066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.489707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.491633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817fc5-441b-4432-83b5-8aa16f6b7354 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.537414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 787/1549] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/projects => generated 391 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:46.538748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2356-3b54-4e2b-aa50-c29989e828f1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.539039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2356-3b54-4e2b-aa50-c29989e828f1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.539324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2356-3b54-4e2b-aa50-c29989e828f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.559917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-25ef2356-3b54-4e2b-aa50-c29989e828f1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.561126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-25ef2356-3b54-4e2b-aa50-c29989e828f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.606316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 763/1550] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:46.607701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388ecc-e8e7-4a50-ae68-4267a8d477d9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.607996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388ecc-e8e7-4a50-ae68-4267a8d477d9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.608280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388ecc-e8e7-4a50-ae68-4267a8d477d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.625302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 788/1551] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.626698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa59dc-d77b-4891-904d-716a041ffe3b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.627036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa59dc-d77b-4891-904d-716a041ffe3b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.631504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa59dc-d77b-4891-904d-716a041ffe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.631626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e7388ecc-e8e7-4a50-ae68-4267a8d477d9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.631916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e7388ecc-e8e7-4a50-ae68-4267a8d477d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.650411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-82fa59dc-d77b-4891-904d-716a041ffe3b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.651591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-82fa59dc-d77b-4891-904d-716a041ffe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.695050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 764/1552] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.696384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.696686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.696972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.713059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 789/1553] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.714515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.714811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.715128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.725256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.726529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.734484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.735284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 765/1554] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.735791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.736003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.746365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e76d3-0441-4875-ae40-593851da4907 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.747311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 790/1555] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.748103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.753223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.754106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.754645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.754933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.756278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:46.756777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.758729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:01:46.758729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 18:01:46.758729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 18:01:46.759737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.759983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2876483-5f91-4508-9990-7e9db6c9ada8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.766610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.767235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.767521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.767800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.768469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.770506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8812f-4e07-4804-b8d2-acf02ce21df9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.773632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 766/1556] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:46.774543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.786220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9b30-093f-4f30-b5fa-2cc5ee1f4936 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.790965 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.792013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.792013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.792728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.792898 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.793916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:01:46.793916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 18:01:46.793916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 18:01:46.795150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cad04e-9d6c-4c12-ba85-af68054c9a4a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.877141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 791/1557] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:46.878480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d87d7-5c46-4215-8a2a-d6a9caf4a31d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.878777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d87d7-5c46-4215-8a2a-d6a9caf4a31d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.879058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d87d7-5c46-4215-8a2a-d6a9caf4a31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.894455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 767/1558] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:46.896205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.896494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.896775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.901727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9e8d87d7-5c46-4215-8a2a-d6a9caf4a31d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.903281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9e8d87d7-5c46-4215-8a2a-d6a9caf4a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.915970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 792/1559] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.915970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776cb7-3cdd-410f-8e19-0450a1e02138 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.915970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776cb7-3cdd-410f-8e19-0450a1e02138 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.915970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776cb7-3cdd-410f-8e19-0450a1e02138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.918750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.920020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.928428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 768/1560] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.932285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.934818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d5776cb7-3cdd-410f-8e19-0450a1e02138 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.936029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d5776cb7-3cdd-410f-8e19-0450a1e02138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.939694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cffac-1475-4591-a6c0-1e4df9917ca8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.944023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 793/1561] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.944737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.945275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.945391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.945567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.945697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.945846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.945943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.946616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:46.946932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.949091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.953550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:46.961340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 769/1562] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:46.962163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.965731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:46.966967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:46.974193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ab59e-fd7b-4bb3-ac36-dd0ddde1f3d7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.975291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 794/1563] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:46.975291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:46.983586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.984215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.984530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.984805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.985471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.987453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:46.989352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb3988-e3f3-45e0-a684-59697ecccf03 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:46.993981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:46.994625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:46.994938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:46.995248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:46.995990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:46.998486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.011234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 795/1564] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:47.012436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.022321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954c0d6-146c-46ba-9e02-c9ef42fa15a3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.027526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.028343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.028506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.028790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.029528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.031209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.047890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 796/1565] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:47.047890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.057445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e5f98-566d-4f2f-8f00-b3aa3d884aba admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.063094 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.063707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.064042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.064327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.065029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:47.065346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.067011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.075561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 770/1566] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:46 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.080686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.083678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 797/1567] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:47.084544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.099119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b4f4-0a40-49a6-a5d0-6612e9da4c8c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.113553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:47.147047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd5d2-7211-4e0d-a121-d1320583f782 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.152320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.153068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.153348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.153626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.154313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:47.154647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.156408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.164701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.165251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.165619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.166011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.166809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:47.167762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.169228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 771/1568] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:47.170081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.171381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.181688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2c8c-5907-41a1-9f5d-9900d0590311 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.187487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 798/1569] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:47.187796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.188499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.188768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.188890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.189053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.189769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.191735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.206367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca6ad6-3962-44b3-9c61-ee128c5215b4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.211415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.212022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.212305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.212645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.213280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:01:47.213578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.215618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.229263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 799/1570] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:01:47.231175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.248604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b521f-ead8-4c72-8f7d-8f6d2ed3c39b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.253452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.254071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.254504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.254793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.255491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.257262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.268329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 800/1571] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:47.269361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.276313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 772/1572] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.276313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.282771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ec65c-7728-428b-96a0-314f2c4cbd30 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.343733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.344394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.344681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.344963 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.345693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.347652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.350891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1628396-6069-47f8-ae9d-f506e3d96e94 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.355635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.356258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.356542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.356820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.357502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.359127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 801/1573] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:47.359533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.363124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.374412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27415-2deb-4cd2-be1d-c31567b790c2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.379047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.384166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.384166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.384166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.384166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.384166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.437079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 773/1574] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.437685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.447518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2df84-5212-4f6e-a69d-b25e21453a61 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.453027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.453623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.453912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.454220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.454916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.456837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.497247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 802/1575] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.497247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.503822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:47.559187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9803-a9ba-4d0a-96a4-7dc1649c4760 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.563884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.564500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.564787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.565067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.565796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.567773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.576523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 774/1576] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.577361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.641753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d15ec1-0a60-41c5-ad9d-ab53091a9b43 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.646374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.647002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.647286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.647565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.648227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.650140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.661856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:47.671540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 803/1577] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.672429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.736868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee50d8-c242-4f1d-a841-6e7b2c87010f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.741566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.742240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.742582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.742881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] PATH_INFO: `/v3/projects/c1c1587cf20d427e8bb03ffae1b1ce33/users/564e0f9000a1448ea9ea37a2892e6495/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.757944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c1587cf20d427e8bb03ffae1b1ce33, user_id=564e0f9000a1448ea9ea37a2892e6495, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.759142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:01:47.759142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 18:01:47.759142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 18:01:47.761955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.766628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 775/1578] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.766628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.828139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893905c-5b4b-476e-979b-230cb25ff86c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.832848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.833474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.833781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.834056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.834750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.836675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.856306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 804/1579] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:47 2022] PUT /identity/v3/projects/c1c1587cf20d427e8bb03ffae1b1ce33/users/564e0f9000a1448ea9ea37a2892e6495/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:47.857412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.920001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1e42-13af-4756-a872-3dfecacf5237 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.924820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.928681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.928681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.928681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] PATH_INFO: `/v3/projects/6f1877c26b444f2489b836a1297c9cb0/users/0c53da1828944c24ae1944ac8781c5f2/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.934426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 776/1580] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:47.934426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.942266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1877c26b444f2489b836a1297c9cb0, user_id=0c53da1828944c24ae1944ac8781c5f2, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.942478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2706581-96b3-4cc4-bf4b-a427b94ffb2f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.945051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.947264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.947867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.948154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.948927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.949069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:47.949753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.951320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.961790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 777/1581] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:47.962256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.973113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56ce8-8adb-4d93-930b-efac2e05218d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.977745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.978373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.981754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.981754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.981754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.981754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.056531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 778/1582] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:47 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:48.066464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 805/1583] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:47 2022] PUT /identity/v3/projects/6f1877c26b444f2489b836a1297c9cb0/users/0c53da1828944c24ae1944ac8781c5f2/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.066663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.066763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.071199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e65c0-13f7-4bc6-9c16-a281a3fe2d57 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.125426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278b5-b5f1-4040-ac89-41c6d0d4f775 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.130281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.130929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.131215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.131431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.131555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.132055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.132227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.132324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.132601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.133318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.134205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.135262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb0745-82c5-43c0-90aa-519f2b0031c3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.218860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 806/1584] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:48.219952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.229557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4afd27-47d1-4510-b04a-2c1f91900157 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.234696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.235340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.235638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.235927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.236604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.238610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.242692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 779/1585] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/users => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:48.246982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.246982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.246982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.264149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:48.265353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:48.272908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 780/1586] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:48.275431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.284179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0f98c-5291-4f36-9a09-bc38f43e0190 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.288721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.289314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.289593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.289866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.290576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.292277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.302570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 781/1587] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.303491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.313598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114442f3-c169-4074-ac0b-453931dfc64b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.318302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.318914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.319195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.321379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.321379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.325338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 807/1588] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:48.325481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e3ef3-8301-417d-ab1b-67704490824f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.325565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.333287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13718ba-4644-49e7-ae63-5a2429edf9af admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.333798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 782/1589] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.335214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.335576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.335795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.338250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.339032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.339587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.339941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.340759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.342370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.353494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 808/1590] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.354327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.363051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:48.365587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07373255-60ea-48b4-b5b7-150370d385c0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.366579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] MFA Rules not processed for user `0c53da1828944c24ae1944ac8781c5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:48.367826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:48.371239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.373012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.373012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.373012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.373442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.375078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.385556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 809/1591] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.387067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.396635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cb8b-6254-401c-81d4-d7d235490c78 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.401347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.402173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.402487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.402762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.403502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.405457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.415519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 810/1592] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.416311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.439742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 783/1593] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:48.441277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.456770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d38412-37f6-4b85-9566-a8aed14f1330 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.463092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.463994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.464293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.464854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.465622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.468160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.476275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c277dc-5c91-4bef-bb0c-96ae8a140085 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.480702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 784/1594] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.480702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.484310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.484310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.484310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.484310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.492852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.493041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd5bce-7b80-474b-a931-757307f4a391 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.494380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.499418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.500045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.500330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.500613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.501324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.502954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.506955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 811/1595] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:48.507575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.513480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 785/1596] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.514292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.520132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b772f20-9393-409d-b880-09b5701f115b service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.524503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ea5a-3c43-4e5c-a0b3-74a9cf3ad8cf admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.525520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.525948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.526250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.526773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] PATH_INFO: `/v3/projects/e993449a4d58431cada94ec21f5a13d9/users/40605f70e0034578acbfc67f7b78b1d0/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.532863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.533487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.533774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.534056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.534784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.536386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.543467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e993449a4d58431cada94ec21f5a13d9, user_id=40605f70e0034578acbfc67f7b78b1d0, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.546377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.547933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 786/1597] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.551336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.561983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1291d-e7a8-4927-9d11-01b1b5eadd2b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.567660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.568184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.568471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.568745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.569426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.571073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.581546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 787/1598] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.582373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.591754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a593a-7574-43d3-ac50-79a6c5df1a3f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.596343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.596932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.597217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.597498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.598179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.599780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88e5f-c6cd-4220-b01b-7175321b54de admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 788/1599] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667983cd-5711-47c9-aa1d-8c8f2a92adca None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667983cd-5711-47c9-aa1d-8c8f2a92adca None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667983cd-5711-47c9-aa1d-8c8f2a92adca None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 789/1600] 10.222.0.85 () {54 vars in 962 bytes} [Thu Jul 28 18:01:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:01:48.613941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667983cd-5711-47c9-aa1d-8c8f2a92adca None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.627314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667983cd-5711-47c9-aa1d-8c8f2a92adca None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.631952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.632654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.632948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.633258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] PATH_INFO: `/v3/projects/d3da48f83596486ca59e9440dd004fb3/users/29ee403697824bf0945236072da0a3aa/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.648639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3da48f83596486ca59e9440dd004fb3, user_id=29ee403697824bf0945236072da0a3aa, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.651352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.677806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 812/1601] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:48 2022] PUT /identity/v3/projects/e993449a4d58431cada94ec21f5a13d9/users/40605f70e0034578acbfc67f7b78b1d0/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.678935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.694869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:48.744081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a7c2-7270-40bb-b3f9-a97a19c6159f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.748763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 790/1602] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:48 2022] PUT /identity/v3/projects/d3da48f83596486ca59e9440dd004fb3/users/29ee403697824bf0945236072da0a3aa/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.749807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.817092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.817789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.818018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.818302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.819036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.820759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.826474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bf76-ebc2-47fa-91ae-02268a5de0d6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.832401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.833236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.833592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.833991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] PATH_INFO: `/v3/projects/2998ec25caee4dab9943ab5ba86ec8eb/users/775115c85a2146f88f49d02d11883154/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.841749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 813/1603] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:48.845923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.857317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f74f5-35c8-4ef0-884a-72b49a91800f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.859306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2998ec25caee4dab9943ab5ba86ec8eb, user_id=775115c85a2146f88f49d02d11883154, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.862435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.862585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.863115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.863404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.863695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] PATH_INFO: `/v3/projects/2732f627abb44936bedd3293898b24f3/users/f740005b7fc245919a6ddbf61ee9ba85/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.881485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2732f627abb44936bedd3293898b24f3, user_id=f740005b7fc245919a6ddbf61ee9ba85, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.884410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f982410-7282-46e7-b176-dcc68dc5a32b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.913331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 791/1604] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:48 2022] PUT /identity/v3/projects/2998ec25caee4dab9943ab5ba86ec8eb/users/775115c85a2146f88f49d02d11883154/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.914309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.926074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:48.936770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 814/1605] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:48 2022] PUT /identity/v3/projects/2732f627abb44936bedd3293898b24f3/users/f740005b7fc245919a6ddbf61ee9ba85/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.938136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190303a4-0b49-43b7-91b6-35fc43ff50ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.938465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190303a4-0b49-43b7-91b6-35fc43ff50ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.938746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190303a4-0b49-43b7-91b6-35fc43ff50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.977682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-190303a4-0b49-43b7-91b6-35fc43ff50ae None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:48.979157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-190303a4-0b49-43b7-91b6-35fc43ff50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:48.987660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e816f6-385d-4e8e-9a22-a4690f9d9328 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.052886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.053729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.054175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 815/1606] 10.222.0.85 () {58 vars in 1059 bytes} [Thu Jul 28 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.055538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ae215-516b-422e-bf24-3bcdd30e8ee8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.055904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ae215-516b-422e-bf24-3bcdd30e8ee8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.056195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ae215-516b-422e-bf24-3bcdd30e8ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.060498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.061203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] PATH_INFO: `/v3/projects/672d7446290f4c2192531e9ab149b966/users/d477c2a2e71c4dc086d2130410f628e5/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.078132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a52ae215-516b-422e-bf24-3bcdd30e8ee8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.079454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a52ae215-516b-422e-bf24-3bcdd30e8ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.082703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672d7446290f4c2192531e9ab149b966, user_id=d477c2a2e71c4dc086d2130410f628e5, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.085425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.087701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 816/1607] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.092601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1f3d-f1c5-4ac5-8011-a094da91c836 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.092897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1f3d-f1c5-4ac5-8011-a094da91c836 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.093371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f1f3d-f1c5-4ac5-8011-a094da91c836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.102591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:49.123500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b68f1f3d-f1c5-4ac5-8011-a094da91c836 None None] MFA Rules not processed for user `29ee403697824bf0945236072da0a3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.127424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b68f1f3d-f1c5-4ac5-8011-a094da91c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.154381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 792/1608] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:48 2022] PUT /identity/v3/projects/672d7446290f4c2192531e9ab149b966/users/d477c2a2e71c4dc086d2130410f628e5/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:49.155072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.213557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 817/1609] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.217339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06a58d-9f32-465a-beb5-ae0a2759f64b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.217649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06a58d-9f32-465a-beb5-ae0a2759f64b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.218055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06a58d-9f32-465a-beb5-ae0a2759f64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.239991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810fc46-f624-47d4-acc5-796bf80cfc25 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.246252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.247281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.253302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.253302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] PATH_INFO: `/v3/projects/c39bbb5307284385a11288bb3246b357/users/9b9c9bfc24d846f294b3ceef65cc0efc/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.256039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ab06a58d-9f32-465a-beb5-ae0a2759f64b None None] MFA Rules not processed for user `775115c85a2146f88f49d02d11883154`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.257288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ab06a58d-9f32-465a-beb5-ae0a2759f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.264658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c39bbb5307284385a11288bb3246b357, user_id=9b9c9bfc24d846f294b3ceef65cc0efc, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.267459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.322442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 818/1610] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.329341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.329341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.329341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.345063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.346276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.354255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 819/1611] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.355095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.363336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 793/1612] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:49 2022] PUT /identity/v3/projects/c39bbb5307284385a11288bb3246b357/users/9b9c9bfc24d846f294b3ceef65cc0efc/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:49.364374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.388458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:49.424467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61678789-8d83-4329-89f7-80e3fd8ba651 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.428449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.429083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.429481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.429672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.456284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f94a15-563e-45d3-b8e4-d8a9beaa88de None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.473074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:49.500195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.506973 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323a4b7-112b-4ae4-a1f9-d0474d734dba service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.513436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 794/1613] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:49.514848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.515071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.515340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.534754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.536024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.544743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 795/1614] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.545855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.546796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.547592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.547881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.548666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.608466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.608698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415eb598-bf18-46bd-b901-7c51f799eb52 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.610297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.612842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.613598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.613892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.614200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.630620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 820/1615] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:49.640157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.655797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dc21b-ad5e-44aa-96ce-3ccc41bbe234 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.662264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.663414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.663886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.664237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.665451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.668143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44dd2-7505-41aa-b34e-e5dae2f76551 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.694050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.695687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.708403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 796/1616] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:49.709281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.719268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7ba8f-7d72-4ee7-bf3e-a9f7ba17aa3a service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.722953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.725390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.725390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.725390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.735918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.737444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.754226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 821/1617] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:49.754226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.754554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.754833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.775293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 797/1618] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:49.783283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:49.800953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:49.811450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] MFA Rules not processed for user `9b9c9bfc24d846f294b3ceef65cc0efc`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:49.811944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fe1f8-f860-4827-946b-258f66db5616 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:49.812860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:49.879480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:49.880549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:49.881052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:49.881521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:49.981168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:49.983356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda990b-cde6-43ad-9399-7aa94e52cc73 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:49.987466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 822/1619] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:49.988631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.009224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 798/1620] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:50.011820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e2ae3-a16e-41e8-a69a-5ccd7ef46b84 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.012114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e2ae3-a16e-41e8-a69a-5ccd7ef46b84 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.012536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e2ae3-a16e-41e8-a69a-5ccd7ef46b84 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.015452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 799/1621] 10.222.0.85 () {54 vars in 912 bytes} [Thu Jul 28 18:01:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:01:50.015799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549549f-7b4e-445a-bfe1-9d16fa49c052 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.022775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25c4a-5e9f-4525-b183-92d4cea06a35 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.023293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25c4a-5e9f-4525-b183-92d4cea06a35 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.023757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d25c4a-5e9f-4525-b183-92d4cea06a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.029160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.034258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.034258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.034258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.034258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.038497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075e4e-596e-42c3-a757-9629c1a3acb4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.048253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-05d25c4a-5e9f-4525-b183-92d4cea06a35 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:50.049512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-05d25c4a-5e9f-4525-b183-92d4cea06a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:50.147799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 823/1622] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:49 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:50.149039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44d6-1879-4120-ac7e-5c622b9efe66 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.149331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44d6-1879-4120-ac7e-5c622b9efe66 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.149965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44d6-1879-4120-ac7e-5c622b9efe66 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.150708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 800/1623] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:01:50.153459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 824/1624] 10.222.0.85 () {54 vars in 962 bytes} [Thu Jul 28 18:01:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:01:50.157330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c44d6-1879-4120-ac7e-5c622b9efe66 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.160961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.160961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.161271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.170784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:50.185639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:50.187161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:50.187415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c44d6-1879-4120-ac7e-5c622b9efe66 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.193394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.193916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.194324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.194554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.195230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.197363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.232954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 801/1625] 10.222.0.85 () {58 vars in 1059 bytes} [Thu Jul 28 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:01:50.234163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.240781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 825/1626] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:50.241672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.279010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e300a-1537-4b64-83b7-99e46586d9b3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.293140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.293871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.294056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.294336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.316003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.317654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.334202 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:50.343505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 826/1627] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:50.344410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.363490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733752c5-0d7c-45d6-ab54-8297928763a1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.379144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc88196-4db9-493b-afc6-20e57f774784 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.390392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.391046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.391334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.391611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.399213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.399943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.403597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.403597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] PATH_INFO: `/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6/users/e3c3d81b0a4e4290a45c076fdf9cea43/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.434229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.438105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.453618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5f007a93fb4462b1a8ee49db0a89f6, user_id=e3c3d81b0a4e4290a45c076fdf9cea43, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.456797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d2f3d-4357-4778-9588-4956aaa907f8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.472091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 802/1628] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:50.568808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 827/1629] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:50 2022] PUT /identity/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6/users/e3c3d81b0a4e4290a45c076fdf9cea43/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:50.573071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.613167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f155709-6c62-486a-ba44-61d74b3a01ce service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.659502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44bfd-32a8-4901-b5fd-09a705227bca None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.659850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44bfd-32a8-4901-b5fd-09a705227bca None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.660143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44bfd-32a8-4901-b5fd-09a705227bca None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.661852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 828/1630] 10.222.0.85 () {54 vars in 912 bytes} [Thu Jul 28 18:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:01:50.669476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.673485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.673485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.703759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:50.703759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:50.747460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.750465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.750832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.751191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.751979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.757478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.780726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 803/1631] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:50.783138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.802533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c2b9-d5c3-428b-9d80-34b9a159d092 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:50.808106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:50.809189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.809712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.810259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] PATH_INFO: `/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6/users/e3c3d81b0a4e4290a45c076fdf9cea43/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.813537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5f007a93fb4462b1a8ee49db0a89f6, user_id=e3c3d81b0a4e4290a45c076fdf9cea43, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:50.816415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfc7f1-9d37-475e-9863-07aba92e254e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:50.842380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 804/1632] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:50 2022] PUT /identity/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6/users/e3c3d81b0a4e4290a45c076fdf9cea43/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:50.853022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:50.853476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:50.853934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:50.861012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 829/1633] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:50.875429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:50.906947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] MFA Rules not processed for user `e3c3d81b0a4e4290a45c076fdf9cea43`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:50.909126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:51.001022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237987-860e-42c3-ac68-6d8371ffbc90 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:51.046081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 805/1634] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:01:51.094405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:51.095473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:51.112501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:51.113042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:51.148053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:51.274752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:51.277796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:51.299234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 830/1635] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:01:52.129948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:52.129948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edca1ab-daa5-4b8d-b731-0fe1fd0a0a6d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:52.129948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:52.129948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:52.141878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:52.141878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:52.141878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:52.141878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165a04-8d3d-4d4a-a141-5cec40e36697 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:52.141878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 806/1636] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:52.526838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:52.536154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6150f4-8181-4cff-9bfd-448d2b7bfd7b service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:52.540591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:52.541465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:52.543444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:52.545367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:52.621344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:52.623339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:52.635638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 831/1637] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:52.756544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:52.756544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:52.756816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:52.807554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] MFA Rules not processed for user `d477c2a2e71c4dc086d2130410f628e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:52.807554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:52.830143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:52.935561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 807/1638] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:01:52.942878 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:53.140544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85005-5ff9-4521-8de6-ec2b01e61b75 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:53.156092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:53.156750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:53.157209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:53.157544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:53.158338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:53.161830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:53.294175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 832/1639] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:01:52 2022] POST /identity/v3/projects => generated 405 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:53.295108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:53.312505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ba1f-dfdd-430e-a053-3f355f55faaa None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:53.564509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:53.565160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:53.565450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:53.565730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:53.566452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:53.568406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:53.733594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:53.736732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 808/1640] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:01:53 2022] POST /identity/v3/users => generated 309 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:01:53.755692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213c34-3904-4e19-870f-7029a485c51d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:53.786491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:53.787564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:53.788067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:53.788556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:53.789836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:53.793780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:53.833754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 833/1641] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:53.838206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:53.880934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd5e-1375-47d2-93de-cc13b28daebb admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:53.895712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:53.896420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:53.896707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:53.896991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] PATH_INFO: `/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af/users/bdf996a9f3e14cf4b46c85e4c558ea15/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:53.930293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed757a22c1d4083af1a7c9cccdfb1af, user_id=bdf996a9f3e14cf4b46c85e4c558ea15, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:53.933445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:54.033043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 809/1642] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:53 2022] PUT /identity/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af/users/bdf996a9f3e14cf4b46c85e4c558ea15/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:54.037341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:54.053022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffffcf8-a458-4a01-95e1-ee9db3cd9893 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:54.082658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:54.209822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:54.210966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:54.211490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:54.212200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:54.213325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:54.215329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e054e5b-ac8a-475f-a0a5-7fc3c43ada0f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:54.247561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 834/1643] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:01:54.253346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:54.280972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade46a6a-33d6-4e9b-a438-7faf3ea5da0c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:54.289316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:54.290397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:54.290907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:54.292982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] PATH_INFO: `/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af/users/bdf996a9f3e14cf4b46c85e4c558ea15/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:54.295734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed757a22c1d4083af1a7c9cccdfb1af, user_id=bdf996a9f3e14cf4b46c85e4c558ea15, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:54.300068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:54.367019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 810/1644] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:01:54 2022] PUT /identity/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af/users/bdf996a9f3e14cf4b46c85e4c558ea15/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:58.459374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7ae6-1f0d-46c2-a956-093fa7a3a152 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:58.459983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7ae6-1f0d-46c2-a956-093fa7a3a152 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:58.459983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7ae6-1f0d-46c2-a956-093fa7a3a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:58.566917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e6da7ae6-1f0d-46c2-a956-093fa7a3a152 None None] MFA Rules not processed for user `bdf996a9f3e14cf4b46c85e4c558ea15`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:01:58.567091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e6da7ae6-1f0d-46c2-a956-093fa7a3a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:58.762260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 835/1645] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:01:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:00.294371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:00.479901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ef17-f697-4f8c-b263-4edb7fb77b42 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:00.494334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:00.495786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:00.496180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:00.497874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] PATH_INFO: `/v3/users/9b9c9bfc24d846f294b3ceef65cc0efc` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:00.502169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9c9bfc24d846f294b3ceef65cc0efc)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:00.506586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:00.562441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:00.873384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9c9bfc24d846f294b3ceef65cc0efc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:00.874636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9c9bfc24d846f294b3ceef65cc0efc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:00.876739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9c9bfc24d846f294b3ceef65cc0efc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:00.914832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9c9bfc24d846f294b3ceef65cc0efc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:01.030536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9c9bfc24d846f294b3ceef65cc0efc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:01.106013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 811/1646] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:00 2022] DELETE /identity/v3/users/9b9c9bfc24d846f294b3ceef65cc0efc => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:01.110685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0d753-72e6-4eb8-bdd6-1424feada204 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.117784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0d753-72e6-4eb8-bdd6-1424feada204 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.117784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0d753-72e6-4eb8-bdd6-1424feada204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.159425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-98f0d753-72e6-4eb8-bdd6-1424feada204 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:01.168369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-98f0d753-72e6-4eb8-bdd6-1424feada204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:01.348994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 836/1647] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:01.421560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:01.432751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:01.478034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c3ff1-f9e2-465e-8e51-7e30c84a77c9 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.478504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c3ff1-f9e2-465e-8e51-7e30c84a77c9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.479812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c3ff1-f9e2-465e-8e51-7e30c84a77c9 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.481399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 837/1648] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:02:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:01.543710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d6328-1e4f-4683-866f-4d0c977ecffa None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.544174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d6328-1e4f-4683-866f-4d0c977ecffa None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.544689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d6328-1e4f-4683-866f-4d0c977ecffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.598905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74708edf-e583-4dc6-89a0-9e0ef2ead0f4 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:01.611468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-229d6328-1e4f-4683-866f-4d0c977ecffa None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:01.613435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:01.614103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.616440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.617185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-229d6328-1e4f-4683-866f-4d0c977ecffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:01.617766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.652230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:01.670716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b7167-f6e2-479d-8c06-a74969d36ece service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:01.709985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 812/1649] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:01.715984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69211c61-2b84-40a6-8165-dd4bc269ead8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.717437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69211c61-2b84-40a6-8165-dd4bc269ead8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.717926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69211c61-2b84-40a6-8165-dd4bc269ead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.759707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:01.795371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-69211c61-2b84-40a6-8165-dd4bc269ead8 None None] MFA Rules not processed for user `40605f70e0034578acbfc67f7b78b1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:01.796813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-69211c61-2b84-40a6-8165-dd4bc269ead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:01.830899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 838/1650] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:01.914955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17ed7-e027-473a-b207-5e81a3d57d28 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.914955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17ed7-e027-473a-b207-5e81a3d57d28 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.914955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17ed7-e027-473a-b207-5e81a3d57d28 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.915679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 839/1651] 10.222.0.85 () {54 vars in 961 bytes} [Thu Jul 28 18:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:01.933629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:01.934070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:01.934462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:01.992455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 813/1652] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:01.999781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:02.004980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:02.006054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:02.006739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:02.007617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:02.071472 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 840/1653] 10.222.0.85 () {58 vars in 1058 bytes} [Thu Jul 28 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:02.082921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] MFA Rules not processed for user `40605f70e0034578acbfc67f7b78b1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:02.086008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:02.086464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:02.113037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffea50-45d7-40fe-89dd-b5ebfbde643e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:02.119294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 814/1654] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:02.136998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:02.139058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:02.139374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:02.139908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:02.149650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:02.149650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:02.178113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:02.184762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:02.211428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:02.241954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 841/1655] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:02.335491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25816238-bf13-4c22-9235-eaf2cbd7aacf None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:02.368013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:02.372060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:02.372549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:02.373025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:02.383940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:02.388076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db224b43-c56f-46c2-9b2f-be016c404131 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:02.404188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 815/1656] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:02.692029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:02.693648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:02.711238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0620f07-1029-4eac-ae2d-fe5786d21d0f service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:02.731986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:02.736933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:02.737389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:02.738039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] PATH_INFO: `/v3/users/0c53da1828944c24ae1944ac8781c5f2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:02.741738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c53da1828944c24ae1944ac8781c5f2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:02.744084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: WARNING py.warnings [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:02:02.744084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: warnings.warn(msg) Jul 28 18:02:02.744084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: Jul 28 18:02:02.754068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:03.008270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459efd-58c5-41eb-8be1-8bf7a0d413c6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.008787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459efd-58c5-41eb-8be1-8bf7a0d413c6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.008882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459efd-58c5-41eb-8be1-8bf7a0d413c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.012116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c53da1828944c24ae1944ac8781c5f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.013122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c53da1828944c24ae1944ac8781c5f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.069478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-ce459efd-58c5-41eb-8be1-8bf7a0d413c6 None None] MFA Rules not processed for user `564e0f9000a1448ea9ea37a2892e6495`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:03.073640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ce459efd-58c5-41eb-8be1-8bf7a0d413c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.200005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c53da1828944c24ae1944ac8781c5f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.248017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 816/1657] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:03.253954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.256615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.257098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.275183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c53da1828944c24ae1944ac8781c5f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.276225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-aefb9196-3195-45c3-9385-c64f571b72e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c53da1828944c24ae1944ac8781c5f2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.308967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 842/1658] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:02 2022] DELETE /identity/v3/users/0c53da1828944c24ae1944ac8781c5f2 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:03.314760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] MFA Rules not processed for user `564e0f9000a1448ea9ea37a2892e6495`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:03.314849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.314849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.314849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.315026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.352854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:03.355762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.405509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 817/1659] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:03.418528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:03.419949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.475987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 843/1660] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:03.477188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:03.480751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.521828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:03.528614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb018d-7653-49c0-8faf-0776faf0fa88 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:03.556625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:03.557998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.558646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.559020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] PATH_INFO: `/v3/projects/c39bbb5307284385a11288bb3246b357` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.561116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c39bbb5307284385a11288bb3246b357)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:03.564243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:03.565752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e9f2e-eea3-4c73-92d4-f93a34702e38 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:03.576347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:03.577005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.577293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.577574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.581589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.602877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:03.605163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:03.609648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:03.623443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 818/1661] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:03.624674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:03.626254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:03.701231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39bbb5307284385a11288bb3246b357', 'request_id': 'req-7220f524-e51a-4f6f-9842-12875004b206'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.710151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39bbb5307284385a11288bb3246b357', 'request_id': 'req-7220f524-e51a-4f6f-9842-12875004b206'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.710590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20295e-b975-4460-adf2-f87c514fc9b4 service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:03.717632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39bbb5307284385a11288bb3246b357', 'request_id': 'req-7220f524-e51a-4f6f-9842-12875004b206'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.793107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:03.794214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.794660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.794951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.812897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] The token cache is being invalidate because project c39bbb5307284385a11288bb3246b357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:03.813958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.814823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7220f524-e51a-4f6f-9842-12875004b206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:03.822220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 844/1662] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:03 2022] DELETE /identity/v3/projects/c39bbb5307284385a11288bb3246b357 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:03.829526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:03.829526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:03.829526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:03.835286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 845/1663] 10.222.0.85 () {54 vars in 912 bytes} [Thu Jul 28 18:02:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:03.836498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:03.844170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.022042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.025359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177480e5-ca97-4761-a61e-75acd5b1a295 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.054647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 819/1664] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:04.056046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ec2a-6ee0-4222-ac51-07eb3b9e06a9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.056491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ec2a-6ee0-4222-ac51-07eb3b9e06a9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.058238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8ec2a-6ee0-4222-ac51-07eb3b9e06a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.060663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb46c9-c8c4-475f-93b0-fa458bf19bbf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:04.071785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:04.073166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.075864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.076272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] PATH_INFO: `/v3/users/d477c2a2e71c4dc086d2130410f628e5` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.078724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d477c2a2e71c4dc086d2130410f628e5)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.084508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.112024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9ca8ec2a-6ee0-4222-ac51-07eb3b9e06a9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:04.115276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9ca8ec2a-6ee0-4222-ac51-07eb3b9e06a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.150981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 820/1665] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:04.160580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a98b0-4d2e-46f8-a70a-b344c484062a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.161560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a98b0-4d2e-46f8-a70a-b344c484062a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.162451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a98b0-4d2e-46f8-a70a-b344c484062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.257586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c32a98b0-4d2e-46f8-a70a-b344c484062a None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:04.263782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c32a98b0-4d2e-46f8-a70a-b344c484062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.267643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 821/1666] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:04.269173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e9c2-d92b-497a-949f-0d62f8aae177 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.269539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e9c2-d92b-497a-949f-0d62f8aae177 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.269887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339e9c2-d92b-497a-949f-0d62f8aae177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.302799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1339e9c2-d92b-497a-949f-0d62f8aae177 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:04.304188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1339e9c2-d92b-497a-949f-0d62f8aae177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.319172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 822/1667] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:04.321481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.326833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.326833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.337801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 823/1668] 10.222.0.85 () {54 vars in 912 bytes} [Thu Jul 28 18:02:04 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:04.339186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:04.341238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.369104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:04.375344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd477c2a2e71c4dc086d2130410f628e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:04.376875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd477c2a2e71c4dc086d2130410f628e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:04.394258 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf7de7-089c-4558-ad8b-1c94620442df None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:04.402119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:04.402805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.403096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.403378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.413673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.415633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.465117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 824/1669] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:04.469104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:04.474071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd477c2a2e71c4dc086d2130410f628e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:04.505860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd477c2a2e71c4dc086d2130410f628e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:04.506862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b0187f7f-96a4-4971-926e-569b7e18ec7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd477c2a2e71c4dc086d2130410f628e5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:04.509990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757996d-2216-4405-8651-90bed4314d12 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:04.520669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:04.522071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.524681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.525188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.526995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:04.527371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.533234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.540085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 846/1670] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:03 2022] DELETE /identity/v3/users/d477c2a2e71c4dc086d2130410f628e5 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:04.544310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.544310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.544532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.571701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 825/1671] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:04.575231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:04.578427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.592156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:04.593557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.600011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762e187-7813-4930-a31c-c9d186ee649a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:04.657181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:04.667245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:04.676463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.676463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.676463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.676463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.676463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.716485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 847/1672] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:04.754606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:04.756689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:04.774813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f49bc-fa99-4773-b734-4dab7a762845 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:04.781134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:04.782201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:04.782728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:04.783198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:04.796292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:04.798158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:04.812230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 848/1673] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:04.873661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 826/1674] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:04.878447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:04.902608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb228913-279b-48ce-8221-dcc47210a7fd service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.037092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787674-7991-4671-af86-6d060b716714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:05.037922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787674-7991-4671-af86-6d060b716714 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.039649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787674-7991-4671-af86-6d060b716714 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.040019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787674-7991-4671-af86-6d060b716714 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.055238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787674-7991-4671-af86-6d060b716714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:05.057904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787674-7991-4671-af86-6d060b716714 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:05.181860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 849/1675] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:04 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:05.182207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:05.196755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043748d1-ccd5-460b-af52-8348b0f82376 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.215166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:05.215816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.216106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.216389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.217109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:05.218789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:05.242427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 827/1676] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:05.245177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787674-7991-4671-af86-6d060b716714 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:05.262712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787674-7991-4671-af86-6d060b716714 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.278257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:05.279057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.279335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.279639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.282380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:05.286335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:05.325298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:05.327574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:05.334396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 850/1677] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:05.345853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:05.370270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fa78c-8619-463e-85ee-480266c7b830 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.377847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926fe00-d5d4-4eec-b677-45576a9d05f2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.399952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:05.399952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.399952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.399952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] PATH_INFO: `/v3/projects/8f3bb1d4e49c4c229d4cb6ed1ddebbf2/users/08bc7ed870fa492396c4489d4b37f0b5/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.400934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:05.400934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.400934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.405632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] PATH_INFO: `/v3/projects/6f1877c26b444f2489b836a1297c9cb0` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.407087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1877c26b444f2489b836a1297c9cb0)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:05.413189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: WARNING py.warnings [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:02:05.413189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: warnings.warn(msg) Jul 28 18:02:05.413189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: Jul 28 18:02:05.414473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:05.435331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3bb1d4e49c4c229d4cb6ed1ddebbf2, user_id=08bc7ed870fa492396c4489d4b37f0b5, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:05.438515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:05.577292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 851/1678] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:05 2022] PUT /identity/v3/projects/8f3bb1d4e49c4c229d4cb6ed1ddebbf2/users/08bc7ed870fa492396c4489d4b37f0b5/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:05.591993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:05.599277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:05.631425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1877c26b444f2489b836a1297c9cb0', 'request_id': 'req-963db320-7c06-4dd3-928a-0f488c0e62f7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:05.661862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1877c26b444f2489b836a1297c9cb0', 'request_id': 'req-963db320-7c06-4dd3-928a-0f488c0e62f7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:05.764950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1877c26b444f2489b836a1297c9cb0', 'request_id': 'req-963db320-7c06-4dd3-928a-0f488c0e62f7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:05.828343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] The token cache is being invalidate because project 6f1877c26b444f2489b836a1297c9cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:05.828343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:05.833095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-963db320-7c06-4dd3-928a-0f488c0e62f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:05.838612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 828/1679] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:05 2022] DELETE /identity/v3/projects/6f1877c26b444f2489b836a1297c9cb0 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:05.845178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:05.845178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:05.845514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:05.873121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32150d47-89bf-48d4-bafd-b189ce97a476 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:05.939099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:05.987544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] MFA Rules not processed for user `08bc7ed870fa492396c4489d4b37f0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:05.989032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.048710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:06.049994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.050521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.051316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] PATH_INFO: `/v3/projects/672d7446290f4c2192531e9ab149b966` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.053902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672d7446290f4c2192531e9ab149b966)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:06.056268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:06.058707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:06.132297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 829/1680] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:06.133154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:06.134583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.221340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672d7446290f4c2192531e9ab149b966', 'request_id': 'req-b2a11238-eadd-448a-9ee4-891c7d59fb54'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:06.229497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672d7446290f4c2192531e9ab149b966', 'request_id': 'req-b2a11238-eadd-448a-9ee4-891c7d59fb54'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:06.237669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672d7446290f4c2192531e9ab149b966', 'request_id': 'req-b2a11238-eadd-448a-9ee4-891c7d59fb54'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:06.261957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da38bf-7ed2-4c6a-a4de-c759df59ba4f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:06.362962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] The token cache is being invalidate because project 672d7446290f4c2192531e9ab149b966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:06.363268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:06.367352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b2a11238-eadd-448a-9ee4-891c7d59fb54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:06.373919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 852/1681] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:05 2022] DELETE /identity/v3/projects/672d7446290f4c2192531e9ab149b966 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:06.380569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c119a-19e9-4630-bfc2-1e596c85f3d4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.381228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c119a-19e9-4630-bfc2-1e596c85f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.381825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c119a-19e9-4630-bfc2-1e596c85f3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.437816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:06.438833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.439524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.440120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.447006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.451659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3c4c119a-19e9-4630-bfc2-1e596c85f3d4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:06.454860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3c4c119a-19e9-4630-bfc2-1e596c85f3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.485495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:06.489263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:06.514651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 830/1682] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:06.516433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:06.522591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.558644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863ea4a-7f96-4b89-ab6b-4a739e289462 service placement] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:06.580807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:06.581460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.581773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.588923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.611224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 853/1683] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:06.617507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.617507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.617507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.626997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:06.632514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:06.664457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] MFA Rules not processed for user `564e0f9000a1448ea9ea37a2892e6495`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:06.667067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.671129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 831/1684] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:06.672877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:06.679470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.829684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 854/1685] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:06.834205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:06.836444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:06.866771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:06.892340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff62caf-e0c2-4c03-b0bd-936edded6ebf None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:06.917794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:06.917794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.917794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.917794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] PATH_INFO: `/v3/users/bdf996a9f3e14cf4b46c85e4c558ea15` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.918161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf996a9f3e14cf4b46c85e4c558ea15)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:06.921294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:06.928691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22555-141a-4c5a-b01e-88e3ec3ca9c5 service placement] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:06.953084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:06.954053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:06.954453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:06.954805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:06.969374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:07.129650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:07.137149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de556ff3-22ba-44a4-bcb0-d234c0318881 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:07.161634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 832/1686] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:07.163059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6e28-326b-41d1-86d0-5404cd6aab01 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:07.163368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6e28-326b-41d1-86d0-5404cd6aab01 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:07.163627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb6e28-326b-41d1-86d0-5404cd6aab01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:07.202228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:07.210127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-eefb6e28-326b-41d1-86d0-5404cd6aab01 None None] MFA Rules not processed for user `f740005b7fc245919a6ddbf61ee9ba85`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:07.212203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-eefb6e28-326b-41d1-86d0-5404cd6aab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:07.264353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf996a9f3e14cf4b46c85e4c558ea15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:07.265175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf996a9f3e14cf4b46c85e4c558ea15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:07.345765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf996a9f3e14cf4b46c85e4c558ea15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:07.389710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 833/1687] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:07.393289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:07.393588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:07.393877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:07.420650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf996a9f3e14cf4b46c85e4c558ea15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:07.441203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf996a9f3e14cf4b46c85e4c558ea15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:07.464689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:07.465954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:07.499344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 834/1688] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:07.505308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 855/1689] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:06 2022] DELETE /identity/v3/users/bdf996a9f3e14cf4b46c85e4c558ea15 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:07.505591 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:07.506491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:07.527525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:07.543267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb85e4-6f99-48e4-b779-8a929e6784f5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:07.695593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:07.731603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:07.733495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:07.733796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:07.735302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:07.743755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:07.757455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6b47c-ee1d-435f-b01e-573443da89ed admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:07.768750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:07.769740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:07.770199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:07.770738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:07.777014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:07.777014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:07.778862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:07.820032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 856/1690] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:07.824773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:07.863442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4494ee-d202-42b6-a246-dc4bdf896762 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:07.878424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:07.880529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:07.880891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:07.881841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:07.883117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:07.887470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.007239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 857/1691] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:07 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:08.008274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.016968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.027210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.064036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 835/1692] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:08.138693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.140943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:08.172721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a0c8-23bc-474a-b41d-784d231db602 service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.178911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:08.202500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d722-438d-413e-9453-9b234b4425a6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.208549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.209412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.210129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.210610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.211594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.215219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.269340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.270019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.270302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.270613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] PATH_INFO: `/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.271825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed757a22c1d4083af1a7c9cccdfb1af)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.274041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.386611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 858/1693] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:08 2022] POST /identity/v3/users => generated 302 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:08.393321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.423442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399491b8-113d-4a72-b1ec-cb187d7e025d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.457574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.458126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.458440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.458720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.459423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.461168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.483532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed757a22c1d4083af1a7c9cccdfb1af', 'request_id': 'req-5f106bad-41ae-4828-a399-e8c985980ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:08.490685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 859/1694] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:08.495062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.497253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed757a22c1d4083af1a7c9cccdfb1af', 'request_id': 'req-5f106bad-41ae-4828-a399-e8c985980ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:08.517142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be34ab-671e-4a48-b6f4-95b852ffbccc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.654234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.656243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.657380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.658389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed757a22c1d4083af1a7c9cccdfb1af', 'request_id': 'req-5f106bad-41ae-4828-a399-e8c985980ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:08.659841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.670170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.691943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.710848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] The token cache is being invalidate because project 6ed757a22c1d4083af1a7c9cccdfb1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:08.711155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:08.712594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5f106bad-41ae-4828-a399-e8c985980ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:08.720019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 836/1695] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:08 2022] DELETE /identity/v3/projects/6ed757a22c1d4083af1a7c9cccdfb1af => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:08.721646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.721938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.722221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.738400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 860/1696] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:08.739664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.748096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:08.786834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:08.789830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:08.793422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b386b3-da8a-4080-b494-82dd46b0e1a8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.806417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.806417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.806417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.806417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] PATH_INFO: `/v3/projects/5ef0b53a544d4bbe8d3ad31b1832b768/users/25d9e202604348d9986d1ea2ab528931/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.864481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef0b53a544d4bbe8d3ad31b1832b768, user_id=25d9e202604348d9986d1ea2ab528931, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.864481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce6e85-9b1b-464d-8bc0-6c9bf4023f7e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:08.865616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 837/1697] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:08.865616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:08.865616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:08.907025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984022de-06aa-43a8-a6f1-f096d2404df1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:08.923133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:08.924033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:08.924455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:08.924877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] PATH_INFO: `/v3/users/08bc7ed870fa492396c4489d4b37f0b5` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:08.926450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bc7ed870fa492396c4489d4b37f0b5)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:08.930841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:09.000676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:09.130168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 861/1698] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:08 2022] PUT /identity/v3/projects/5ef0b53a544d4bbe8d3ad31b1832b768/users/25d9e202604348d9986d1ea2ab528931/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:09.146126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c1d08-fef2-4ad7-9caa-45c38081a1ac None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:09.147551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c1d08-fef2-4ad7-9caa-45c38081a1ac None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:09.147990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c1d08-fef2-4ad7-9caa-45c38081a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:09.225984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d62c1d08-fef2-4ad7-9caa-45c38081a1ac None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:09.225984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d62c1d08-fef2-4ad7-9caa-45c38081a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:09.294843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 862/1699] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:09.296578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:09.297334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:09.297873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:09.336993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] MFA Rules not processed for user `25d9e202604348d9986d1ea2ab528931`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:09.339057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:09.378037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:09.395511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bc7ed870fa492396c4489d4b37f0b5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.398556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bc7ed870fa492396c4489d4b37f0b5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.400690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bc7ed870fa492396c4489d4b37f0b5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.446610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bc7ed870fa492396c4489d4b37f0b5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.465882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 863/1700] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:09.465882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:09.489705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:09.499914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc4cf4-d35c-4bdb-8f90-248552bea86b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:09.528996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:09.532737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:09.532737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:09.532737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] PATH_INFO: `/v3/users/29ee403697824bf0945236072da0a3aa` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:09.534501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ee403697824bf0945236072da0a3aa)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:09.536492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:09.727781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bc7ed870fa492396c4489d4b37f0b5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.798193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 838/1701] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:08 2022] DELETE /identity/v3/users/08bc7ed870fa492396c4489d4b37f0b5 => generated 0 bytes in 950 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:09.803975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:09.812666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:09.891613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ee403697824bf0945236072da0a3aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.892053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ee403697824bf0945236072da0a3aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:09.973931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a9b44-9ec7-435a-87cb-2b125d023831 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:09.980978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:09.981798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:09.982241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:09.982696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:09.983709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:09.984165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:09.987216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.004233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 839/1702] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:10.005529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.012672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.020534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ee403697824bf0945236072da0a3aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:10.048823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd98c3-a34d-4a5e-9175-2e8f3ce4ef14 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.052788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ee403697824bf0945236072da0a3aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:10.053868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b76c32f-73d4-451e-855c-6e4f94441054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ee403697824bf0945236072da0a3aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:10.070126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.070957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.071257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.071542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.075462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.082877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.099398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 864/1703] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:09 2022] DELETE /identity/v3/users/29ee403697824bf0945236072da0a3aa => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:10.103139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed04d-3d24-4c92-bf2d-6be22c6f06c1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.103658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed04d-3d24-4c92-bf2d-6be22c6f06c1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.104096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed04d-3d24-4c92-bf2d-6be22c6f06c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.130681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d99ed04d-3d24-4c92-bf2d-6be22c6f06c1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:10.132364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d99ed04d-3d24-4c92-bf2d-6be22c6f06c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.208439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 840/1704] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/projects => generated 405 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:10.209642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.213624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.272933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 865/1705] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:10.272933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.272933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.272933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.304515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:10.306305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.329404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41f183-1a25-4d94-a539-5e6ad167e2b8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.342594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.346810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.346810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.346810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.346810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.346810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.463411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 841/1706] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/projects => generated 371 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:10.464766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.470788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 866/1707] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:10.471841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.472618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.474198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:10.516318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:10.608177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:10.615509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18398b68-dea6-41c6-a9e8-5dc3f5be6959 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.627799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458b168-a5ac-4d9c-86d3-a0fb7bad919d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.638932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.640088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.640386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.640934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.641410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.643597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.725812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.726779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.728371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.729427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.747521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.749662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.764507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 867/1708] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:10.765538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.780148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c5876-52c6-4cfc-a419-67bb3dfcf84e service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.800705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 842/1709] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/users => generated 309 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:10.801655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.807718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.807718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.807718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.807718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.808147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.809808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.847542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44929-c3c6-4bbb-818d-8cd6bc0bfd6f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:10.858451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:10.859545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:10.859949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:10.860360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:10.885459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:10.890961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:10.945665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 843/1710] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:10.946555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:10.974884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18085-a087-4c1b-a189-219f380e43b0 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.006592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.007902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.008191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.008473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.009198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.015857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 868/1711] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:10 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:11.018579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.022753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.047670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdb5ee-f902-4fb2-ac4b-8da83316cefa admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.065453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 844/1712] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:11.067710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.070274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.072264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.073665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.074886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.077790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.086886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.095085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde93a-a0eb-4ec9-bd1a-ebeacf2b3fe3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.107426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 869/1713] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:11.108791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.120246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.121458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.121846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.122300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] PATH_INFO: `/v3/projects/98fb3fb143c5450aa456139881675d09/users/b2e6b8db5fe34813994ecdee617ea469/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.137204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0bebf-584d-42d7-9b46-9c65d72288f6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.173017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.175198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.177226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.181541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.181541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.184902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.187615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fb3fb143c5450aa456139881675d09, user_id=b2e6b8db5fe34813994ecdee617ea469, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.193822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.227227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 870/1714] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:11.234144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.247686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2363a0-22aa-4036-b4fd-f93cc9e8138a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.254667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.255420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.256233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.256519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] PATH_INFO: `/v3/projects/9da75e79415f49079b188a7077e06729/users/0c5e2ea8311b4eb78963fddd85f15270/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.275239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 845/1715] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:11 2022] PUT /identity/v3/projects/98fb3fb143c5450aa456139881675d09/users/b2e6b8db5fe34813994ecdee617ea469/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:11.279359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.310541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da75e79415f49079b188a7077e06729, user_id=0c5e2ea8311b4eb78963fddd85f15270, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.317456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02d92-1617-44c6-a452-5df773894658 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.387579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1e0a-423a-42f0-9d05-60198dbbc4ee admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.392735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.393386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.393711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.394134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.395106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.397967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.418475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 846/1716] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:11.422072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.433698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6225d-764c-4ec2-bc6b-28c7edfc842c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.448108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 871/1717] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:11 2022] PUT /identity/v3/projects/9da75e79415f49079b188a7077e06729/users/0c5e2ea8311b4eb78963fddd85f15270/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:11.464399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.465856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.467189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.573048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.573765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.574059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.575029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] PATH_INFO: `/v3/projects/98fb3fb143c5450aa456139881675d09/users/b2e6b8db5fe34813994ecdee617ea469/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.576493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] MFA Rules not processed for user `0c5e2ea8311b4eb78963fddd85f15270`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:11.579869 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fb3fb143c5450aa456139881675d09, user_id=b2e6b8db5fe34813994ecdee617ea469, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.581378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:11.584305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0be34-c784-47cc-879d-52d9a167fe63 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.610986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 847/1718] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:11 2022] PUT /identity/v3/projects/98fb3fb143c5450aa456139881675d09/users/b2e6b8db5fe34813994ecdee617ea469/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:11.623842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.624265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.624714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.685570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] MFA Rules not processed for user `b2e6b8db5fe34813994ecdee617ea469`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:11.686886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:11.751376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:11.756162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 872/1719] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:11.757131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.771377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcc9dc-b5b2-4e58-bdde-12fe7fc3c45f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:11.874646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 848/1720] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:11.875091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:11.879472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:11.894048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:11.907059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:11.908258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:11.908545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:11.908822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:11.909791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:11.912411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:11.997626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2475f6-7a01-4755-902f-68df4708d8d1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:12.002482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:12.013370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.014689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.015047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] PATH_INFO: `/v3/projects/8f3bb1d4e49c4c229d4cb6ed1ddebbf2` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:12.016569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3bb1d4e49c4c229d4cb6ed1ddebbf2)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:12.020004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:12.060458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 873/1721] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:11 2022] POST /identity/v3/projects => generated 371 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:12.061244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:12.067390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:12.215923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3bb1d4e49c4c229d4cb6ed1ddebbf2', 'request_id': 'req-a61ab010-ca25-4e87-92bf-c6dd31048d65'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.218953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c8bd-1cf6-4d68-b684-4566399d6a1d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:12.232432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3bb1d4e49c4c229d4cb6ed1ddebbf2', 'request_id': 'req-a61ab010-ca25-4e87-92bf-c6dd31048d65'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.359565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:12.360557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.361538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.362817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:12.370573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3bb1d4e49c4c229d4cb6ed1ddebbf2', 'request_id': 'req-a61ab010-ca25-4e87-92bf-c6dd31048d65'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.387934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] The token cache is being invalidate because project 8f3bb1d4e49c4c229d4cb6ed1ddebbf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:12.388241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.393185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.398078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 849/1722] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:11 2022] DELETE /identity/v3/projects/8f3bb1d4e49c4c229d4cb6ed1ddebbf2 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:12.398078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:12.401384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:12.455436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:12.457037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:12.469673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 874/1723] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:12.470812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:12.605044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ab010-ca25-4e87-92bf-c6dd31048d65 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:12.623447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:12.627799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.627799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.627799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] PATH_INFO: `/v3/projects/d3da48f83596486ca59e9440dd004fb3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:12.627799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3da48f83596486ca59e9440dd004fb3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:12.629074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:12.685509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925e6f2-b68a-4826-a013-e3f4df91cb21 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:12.696507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:12.698340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.698666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.701460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:12.702295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:12.706658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8beba7d-4220-4da9-9b15-d1dbed77694c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:12.795579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:12.816495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 875/1724] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:12 2022] POST /identity/v3/users => generated 292 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:12.818601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3da48f83596486ca59e9440dd004fb3', 'request_id': 'req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.828289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.835117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.835117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:12.835325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3da48f83596486ca59e9440dd004fb3', 'request_id': 'req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.877959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:12.879417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:12.929472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3da48f83596486ca59e9440dd004fb3', 'request_id': 'req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.947647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] The token cache is being invalidate because project d3da48f83596486ca59e9440dd004fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:12.947995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.950552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-85ab7bf7-ee20-4890-a1ad-24589e52a4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:12.954776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 850/1725] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:12 2022] DELETE /identity/v3/projects/d3da48f83596486ca59e9440dd004fb3 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:12.957991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:12.957991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:12.957991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.049291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 876/1726] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.049291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:13.052745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.087055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:13.089428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a2414-4db6-495b-9ef3-2783a56d1fdd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:13.107553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:13.108266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.109384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.109384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] PATH_INFO: `/v3/users/25d9e202604348d9986d1ea2ab528931` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.110582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d9e202604348d9986d1ea2ab528931)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:13.112432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:13.129995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:13.131757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.225613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 851/1727] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.226913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:13.229277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.251865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f51423-38b7-47b2-bd58-dfcee02dbc93 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:13.256980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:13.257650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.257938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.258218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] PATH_INFO: `/v3/users/775115c85a2146f88f49d02d11883154` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.260123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=775115c85a2146f88f49d02d11883154)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:13.262438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:13.379371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d9e202604348d9986d1ea2ab528931'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.380972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d9e202604348d9986d1ea2ab528931'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.458459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d9e202604348d9986d1ea2ab528931'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.477073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d9e202604348d9986d1ea2ab528931'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.478812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d9e202604348d9986d1ea2ab528931'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.496164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 877/1728] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:13 2022] DELETE /identity/v3/users/25d9e202604348d9986d1ea2ab528931 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:13.497642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:13.499793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.517931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775115c85a2146f88f49d02d11883154'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.518731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775115c85a2146f88f49d02d11883154'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.519408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775115c85a2146f88f49d02d11883154'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.537786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775115c85a2146f88f49d02d11883154'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.591846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8629-1de2-4ad4-87f1-7cd2d0cd5111 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:13.601997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:13.602677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.602969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.603320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.604449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:13.605659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e47a-06f7-43d0-af01-9d4633fc2079 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:13.621014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 878/1729] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:13.622532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefe0b-53c0-4184-9515-4e0302bd0c51 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.622831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefe0b-53c0-4184-9515-4e0302bd0c51 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.623113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefe0b-53c0-4184-9515-4e0302bd0c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.644678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bdcefe0b-53c0-4184-9515-4e0302bd0c51 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:13.646564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bdcefe0b-53c0-4184-9515-4e0302bd0c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.657701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 879/1730] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.659376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd327-78c6-4700-86b6-31850422fc94 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.660012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd327-78c6-4700-86b6-31850422fc94 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.660614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd327-78c6-4700-86b6-31850422fc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.669545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775115c85a2146f88f49d02d11883154'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:13.699648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-978dd327-78c6-4700-86b6-31850422fc94 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:13.701044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-978dd327-78c6-4700-86b6-31850422fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.717119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 880/1731] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.717119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b71a-8a9f-4b8e-b41e-ae431a7ca0ab None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.717119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b71a-8a9f-4b8e-b41e-ae431a7ca0ab None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.717119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b71a-8a9f-4b8e-b41e-ae431a7ca0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.735531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 852/1732] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:13 2022] DELETE /identity/v3/users/775115c85a2146f88f49d02d11883154 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:13.736469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:13.739045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.745262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8272b71a-8a9f-4b8e-b41e-ae431a7ca0ab None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:13.750225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8272b71a-8a9f-4b8e-b41e-ae431a7ca0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.781601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:13.856709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 881/1733] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.858587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.859166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.859700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.874867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13eb34-0b03-4472-bf7e-97b053b93d8e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:13.880778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:13.881482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:13.881773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:13.882241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] PATH_INFO: `/v3/projects/9344d2690ed845f88d15ed0892a57250/users/de84a1eca7064880880e9c0f2dda43d3/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:13.891433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:13.894770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:13.907880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9344d2690ed845f88d15ed0892a57250, user_id=de84a1eca7064880880e9c0f2dda43d3, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:13.910945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:13.921355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 882/1734] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:13.922585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:13.924754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.036889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 853/1735] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:13 2022] PUT /identity/v3/projects/9344d2690ed845f88d15ed0892a57250/users/de84a1eca7064880880e9c0f2dda43d3/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:14.036889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.065933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75537-d633-4b68-a134-39c19b009e7f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.130806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27a0c5-d159-4c6f-9cda-ff25be12f262 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.141279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.147020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.147020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.147020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.147020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:14.147020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.149461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b16b-77c7-4b3d-bacf-44b65644686d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.169271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 854/1736] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:14.171529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.172423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.172877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.188386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.189212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.189594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.190017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.192773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.203647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:14.206497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.231552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 855/1737] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:14.232538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.241330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.289241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.291145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.300992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 883/1738] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:02:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:02:14.302276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.348696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcaece-bcb7-41dd-8f83-a604050ca859 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.358842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.359536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.359827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.360116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.362278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9b801-4a67-41e7-93c0-0a194ab12ca3 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.376388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.377161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.379681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.380134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.381169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:14.381486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.383011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.383372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.388412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7c53d-0bb8-441b-bab6-8d12ae0d6d7f service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.395760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:14.421992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 856/1739] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:14.424401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5190-1005-408a-a770-2c30053d250a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.424523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5190-1005-408a-a770-2c30053d250a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.424940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5190-1005-408a-a770-2c30053d250a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.427291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 884/1740] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:14.436604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.453672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b13a5190-1005-408a-a770-2c30053d250a None None] MFA Rules not processed for user `f740005b7fc245919a6ddbf61ee9ba85`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:14.453672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b13a5190-1005-408a-a770-2c30053d250a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.476423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d539-96ae-4b88-9b04-43e901aa2188 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.499959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.500617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.500911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.501201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.501929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.515069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.553883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 885/1741] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:14.560992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.571066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 857/1742] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:14.571066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5190-1005-408a-a770-2c30053d250a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.589796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf85915-0235-433d-955f-afd4844259e9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.601935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5190-1005-408a-a770-2c30053d250a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:14.602916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.604824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.605188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.608664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.608664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.611857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.614559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:14.615205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.615495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.615777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.641397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:14.643905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f625018-9044-4e36-a625-e98eb879f327 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:14.664829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 858/1743] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:14.668260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:14.669109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:14.669752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:14.752226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 886/1744] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:14 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:14.757290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:14.803323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:14.804861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:14.856876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:14.992029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 859/1745] 10.222.0.99 () {58 vars in 1015 bytes} [Thu Jul 28 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:14.993132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.030435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93283-88fe-4229-9b76-52b6fb47e01d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.053396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.053396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.053396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.067951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.068812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.080817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.213729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d85a5-d367-435f-a812-3f148d80ffef None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.223602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.224357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.224647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.224931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] PATH_INFO: `/v3/projects/9344d2690ed845f88d15ed0892a57250/users/de84a1eca7064880880e9c0f2dda43d3/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.227070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9344d2690ed845f88d15ed0892a57250, user_id=de84a1eca7064880880e9c0f2dda43d3, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.233108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.237543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 887/1746] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:14 2022] POST /identity/v3/projects => generated 371 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:15.238307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.282293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 860/1747] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:14 2022] PUT /identity/v3/projects/9344d2690ed845f88d15ed0892a57250/users/de84a1eca7064880880e9c0f2dda43d3/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:15.282293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.285586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:15.311905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:15.429236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0a348-ade0-4375-93c0-12a94922e418 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.488639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a62b-1d67-4e63-8812-2179700ab266 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.520864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.521530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.524682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.525126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.542082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.543428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.555207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.556919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.558102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.558629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.559523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.561957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb8a48-ddb2-479f-944a-4f2d8a9bd22f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.583203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 861/1748] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:15.583827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.609119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d4bad-1a20-4b68-b53d-ebf32115b322 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.622806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.624148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.626940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.626940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.634995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.636619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.651433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 862/1749] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:15.653731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.675775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db34c6-0aae-4108-a5f7-c2f944168ac0 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.681989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.683118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.684052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.684666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.685652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.688299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.701382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 888/1750] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:15 2022] POST /identity/v3/users => generated 289 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:15.708685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.709555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.710401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.781986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] MFA Rules not processed for user `de84a1eca7064880880e9c0f2dda43d3`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:15.783266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:15.798829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 863/1751] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:15 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:15.801790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.807969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:15.823882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8834-79cc-48fc-9ad3-aa30d83fd14a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.830207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.832166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.832985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.833612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.856373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.858306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.874674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 864/1752] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:15.877375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.898380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96ac34-b088-4a25-a5f4-e239da644b79 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.905487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.914021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.914021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.914021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.914021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.914021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:15.933665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 865/1753] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:15.936338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:15.954033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efbeea-005f-4d11-b531-ef3bd5c1f040 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:15.963744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:15.964504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:15.964886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:15.965819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:15.967264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:15.991720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.010729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 866/1754] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:16.011656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.028869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 889/1755] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:16.028869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.030881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b83a-e0d6-4df8-8ded-b12984aea102 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.036717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:16.037388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.037678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.037979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.039083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:16.042549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.161128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8a8bd-5f7a-482c-bbb2-ce8b23018f68 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.161244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 867/1756] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:16.161244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.161244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:16.161478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:16.161478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.161478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.161478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.161814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:16.161814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.161814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 890/1757] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:16.161814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.161814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcd8a-aa89-405b-8441-0fa5b7dff30c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.172797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:16.173946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.174447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.178597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] PATH_INFO: `/v3/projects/c480fcb2f0fe427d860c83be84737011/users/f3518daa855a4fff8839b5094fa61514/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.198626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aced484-05c6-4228-b4c5-6fd03e29822d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.238279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:16.238279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.238279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.238279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.239507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:16.239507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.239669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c480fcb2f0fe427d860c83be84737011, user_id=f3518daa855a4fff8839b5094fa61514, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:16.239765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 868/1758] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:16.239849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.239940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.254291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738349-53d8-4fee-a425-3e0bc24ba33d service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.269618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:16.270215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.270618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.270910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] PATH_INFO: `/v3/projects/6fcd512718fc4562b987412daea3767a/users/369d649af705470081765c82affb448d/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.299359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fcd512718fc4562b987412daea3767a, user_id=369d649af705470081765c82affb448d, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:16.308041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b486f-0802-44be-ae8c-bfe273e065f6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:16.578034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 891/1759] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:16 2022] PUT /identity/v3/projects/c480fcb2f0fe427d860c83be84737011/users/f3518daa855a4fff8839b5094fa61514/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:16.580394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:16.584662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:16.614440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:16.645186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 869/1760] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:16 2022] PUT /identity/v3/projects/6fcd512718fc4562b987412daea3767a/users/369d649af705470081765c82affb448d/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:16.664355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873627-78cb-4a42-a6a9-51c5e1b06824 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:16.666794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873627-78cb-4a42-a6a9-51c5e1b06824 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:16.672512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873627-78cb-4a42-a6a9-51c5e1b06824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:16.810139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:16.848574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-98873627-78cb-4a42-a6a9-51c5e1b06824 None None] MFA Rules not processed for user `f3518daa855a4fff8839b5094fa61514`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:16.859577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-98873627-78cb-4a42-a6a9-51c5e1b06824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:16.957045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab8bbb-a8b1-4670-8785-7247408b5105 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:16.981675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:17.090303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:17.096770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.096770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.096770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] PATH_INFO: `/v3/projects/5ef0b53a544d4bbe8d3ad31b1832b768` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.096770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef0b53a544d4bbe8d3ad31b1832b768)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:17.096770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:17.163879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 870/1761] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:17.166169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.168540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.168540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.254070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:17.255661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.265660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef0b53a544d4bbe8d3ad31b1832b768', 'request_id': 'req-7866e679-b1d2-46b7-b110-35e2fbec9803'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:17.291920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef0b53a544d4bbe8d3ad31b1832b768', 'request_id': 'req-7866e679-b1d2-46b7-b110-35e2fbec9803'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:17.308698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef0b53a544d4bbe8d3ad31b1832b768', 'request_id': 'req-7866e679-b1d2-46b7-b110-35e2fbec9803'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:17.395165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] The token cache is being invalidate because project 5ef0b53a544d4bbe8d3ad31b1832b768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:17.397219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:17.397219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:17.403774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 892/1762] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:16 2022] DELETE /identity/v3/projects/5ef0b53a544d4bbe8d3ad31b1832b768 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:17.404101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:17.405761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.481170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 871/1763] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:17.493069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:17.495065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:17.515725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.550531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ce607-8986-42fe-855b-3bf50ca22fae None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:17.556370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:17.558294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.558294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.558294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.558576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:17.561395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:17.566206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e679-b1d2-46b7-b110-35e2fbec9803 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:17.570718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:17.572277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.572588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.572872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.574959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.709468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 872/1764] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:17 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:17.710697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:17.714548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.738968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:17.742061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:17.756442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:17.758811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 893/1765] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:17.761001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:17.763121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:17.796246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb2ab4-b8e6-492c-a1f6-97e3cfe03a64 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:17.910751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f151b-672c-4582-b3d2-19eb5d3d3512 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:17.921556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:17.922244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.922871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.923163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] PATH_INFO: `/v3/projects/2998ec25caee4dab9943ab5ba86ec8eb` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.939711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2998ec25caee4dab9943ab5ba86ec8eb)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:17.941883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:17.977013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:17.979263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:17.979740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:17.980233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] PATH_INFO: `/v3/users/b2e6b8db5fe34813994ecdee617ea469` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:17.983142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e6b8db5fe34813994ecdee617ea469)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:17.986655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:18.185084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2998ec25caee4dab9943ab5ba86ec8eb', 'request_id': 'req-58146ec4-385b-4103-86a0-754d03becd10'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.191285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2998ec25caee4dab9943ab5ba86ec8eb', 'request_id': 'req-58146ec4-385b-4103-86a0-754d03becd10'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.255398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2998ec25caee4dab9943ab5ba86ec8eb', 'request_id': 'req-58146ec4-385b-4103-86a0-754d03becd10'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.288061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] The token cache is being invalidate because project 2998ec25caee4dab9943ab5ba86ec8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:18.294635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.294635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.299398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 873/1766] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:17 2022] DELETE /identity/v3/projects/2998ec25caee4dab9943ab5ba86ec8eb => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:18.300584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:18.302863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:18.402276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e6b8db5fe34813994ecdee617ea469'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.404317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e6b8db5fe34813994ecdee617ea469'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.494284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58146ec4-385b-4103-86a0-754d03becd10 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:18.504012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:18.504774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:18.505069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:18.505358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:18.515966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:18.575239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e6b8db5fe34813994ecdee617ea469'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.597165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e6b8db5fe34813994ecdee617ea469'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.598560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ff5a4888-3cae-4707-8317-3b1c5b331598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e6b8db5fe34813994ecdee617ea469'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:18.621556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:18.623221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:18.643316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 874/1767] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:18.644641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:18.645536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 894/1768] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:17 2022] DELETE /identity/v3/users/b2e6b8db5fe34813994ecdee617ea469 => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:18.648997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:18.649286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:18.649400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:18.649571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:18.657821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 895/1769] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:02:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:18.657821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:18.666732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:18.691676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65672cdd-6de9-4813-871a-ca16df53fd75 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:18.860006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:18.863924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:18.864218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:18.864504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:18.873694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:18.873694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada64f1-592a-4703-9991-050fad119b34 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:18.881345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d330ad-6936-4ce6-be94-36797451c6d8 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:18.889597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:18.890494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:18.890782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:18.891064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] PATH_INFO: `/v3/users/e3c3d81b0a4e4290a45c076fdf9cea43` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:18.892370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c3d81b0a4e4290a45c076fdf9cea43)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:18.895890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:18.939449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:19.068609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 875/1770] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:18 2022] POST /identity/v3/users => generated 295 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:19.072710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e59d8-72b7-4971-82cb-a0ed3165a78d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.073669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e59d8-72b7-4971-82cb-a0ed3165a78d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.075343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e59d8-72b7-4971-82cb-a0ed3165a78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.114581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-496e59d8-72b7-4971-82cb-a0ed3165a78d None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:19.116304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-496e59d8-72b7-4971-82cb-a0ed3165a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:19.235566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:19.326838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c3d81b0a4e4290a45c076fdf9cea43'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:19.329684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 876/1771] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:02:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:19.330239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c3d81b0a4e4290a45c076fdf9cea43'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:19.331476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.331868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.332257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.333929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 877/1772] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:19.336799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:19.343960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:19.374243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21be2-7a45-4a9c-8495-cf6a7aa936d5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:19.387884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:19.388508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.389233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.389233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.389751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:19.394476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:19.439820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 878/1773] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:19.440770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:19.448414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:19.507802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa80a2-6a79-43dd-8f53-5a591a8989c6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:19.523678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:19.524319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.525179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.531559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.548190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:19.550279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2bab3-e1c5-4565-8a20-dc5e7fa4ef51 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:19.593542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 879/1774] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:19.623168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.624714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.625078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.737558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:19.740821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:19.766777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c3d81b0a4e4290a45c076fdf9cea43'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:19.777169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 880/1775] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:02:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:19.780179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:19.811334 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c3d81b0a4e4290a45c076fdf9cea43'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:19.812986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c3d81b0a4e4290a45c076fdf9cea43'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:19.814519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4afd8-2f2c-4cf9-8a6f-ba49eda1b999 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:19.849210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:19.852125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:19.852417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:19.852738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] PATH_INFO: `/v3/projects/e24320d28ddf494fbcc0aef7310378a1/users/e7d3ae232fa54bc29b28bee275f7a2ec/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:19.893030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 896/1776] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:18 2022] DELETE /identity/v3/users/e3c3d81b0a4e4290a45c076fdf9cea43 => generated 0 bytes in 1236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:19.913032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:19.917409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:19.932602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24320d28ddf494fbcc0aef7310378a1, user_id=e7d3ae232fa54bc29b28bee275f7a2ec, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:19.947314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:20.067711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 881/1777] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:19 2022] PUT /identity/v3/projects/e24320d28ddf494fbcc0aef7310378a1/users/e7d3ae232fa54bc29b28bee275f7a2ec/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:20.074209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:20.079001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:20.101449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964082d-11a0-4d5c-908d-69307f2d2795 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:20.184778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:20.185883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:20.187708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:20.188157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:20.191477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:20.220598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:20.224107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:20.254436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 897/1778] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:20.262470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8daaca-4af5-43e4-9c06-cfa779684494 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:20.302665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:20.302665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:20.302665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:20.302665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:20.303265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:20.303265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8d60-a85a-4cf9-94d3-f9f8ef9ecf25 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:20.341782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 882/1779] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:20.345841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:20.372115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383a3b1-712e-412b-8883-c3cacc747c6b service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:20.383643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:20.385763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:20.386470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:20.386961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] PATH_INFO: `/v3/projects/e24320d28ddf494fbcc0aef7310378a1/users/e7d3ae232fa54bc29b28bee275f7a2ec/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:20.390830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24320d28ddf494fbcc0aef7310378a1, user_id=e7d3ae232fa54bc29b28bee275f7a2ec, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:20.393897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:20.421999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 898/1780] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:20 2022] PUT /identity/v3/projects/e24320d28ddf494fbcc0aef7310378a1/users/e7d3ae232fa54bc29b28bee275f7a2ec/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:20.437019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:20.437713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:20.438234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:20.542702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] MFA Rules not processed for user `e7d3ae232fa54bc29b28bee275f7a2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:20.550447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:20.913426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 883/1781] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:20.925769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:20.928349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.039422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f088-1b8e-4b61-aebe-d27ce96e4f4d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:21.044527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:21.045488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.047199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.048464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.051380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.144771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:21.146638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:21.198709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 899/1782] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:21.198890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:21.198890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.295851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4daed-b220-46b2-b9f4-976cf0b6def9 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:21.301132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:21.301855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.302155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.302467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] PATH_INFO: `/v3/projects/98fb3fb143c5450aa456139881675d09` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.304722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fb3fb143c5450aa456139881675d09)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:21.307683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:21.545544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fb3fb143c5450aa456139881675d09', 'request_id': 'req-255d7c47-eeb9-4eed-9ae5-30892aa2095f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.553125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fb3fb143c5450aa456139881675d09', 'request_id': 'req-255d7c47-eeb9-4eed-9ae5-30892aa2095f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.620114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:21.620114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.635847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e274-95ac-40b6-a071-46c301d77a18 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:21.695294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fb3fb143c5450aa456139881675d09', 'request_id': 'req-255d7c47-eeb9-4eed-9ae5-30892aa2095f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.721920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] The token cache is being invalidate because project 98fb3fb143c5450aa456139881675d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:21.728968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.728968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-255d7c47-eeb9-4eed-9ae5-30892aa2095f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.728968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 884/1783] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:21 2022] DELETE /identity/v3/projects/98fb3fb143c5450aa456139881675d09 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:21.736063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78ffb-f146-4fc6-b920-7d754bd91bc8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.736570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78ffb-f146-4fc6-b920-7d754bd91bc8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.737134 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78ffb-f146-4fc6-b920-7d754bd91bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.778846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:21.779569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.781297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.781641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] PATH_INFO: `/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.784555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5f007a93fb4462b1a8ee49db0a89f6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:21.789632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:21.794410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-03e78ffb-f146-4fc6-b920-7d754bd91bc8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:21.796354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-03e78ffb-f146-4fc6-b920-7d754bd91bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.817031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 885/1784] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:21.832538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.832842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.833326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.881749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:21.898631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:21.915003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 886/1785] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:21.934793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:21.938443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bbab8-f51b-4825-8fcc-a2ba3037df98 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:21.945040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:21.945891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:21.946247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:21.947311 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:21.948131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:21.948511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:21.950711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:21.988200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5f007a93fb4462b1a8ee49db0a89f6', 'request_id': 'req-245b2cf2-238f-42f0-9da0-10b62ee30a31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:21.988717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 887/1786] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:21.989836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:22.005781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5f007a93fb4462b1a8ee49db0a89f6', 'request_id': 'req-245b2cf2-238f-42f0-9da0-10b62ee30a31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:22.017724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543ab8-1221-441a-9922-dce1b98d87b6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:22.042320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5f007a93fb4462b1a8ee49db0a89f6', 'request_id': 'req-245b2cf2-238f-42f0-9da0-10b62ee30a31'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:22.169989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:22.170884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:22.171363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:22.171816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:22.172976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:22.176001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:22.196101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] The token cache is being invalidate because project 1f5f007a93fb4462b1a8ee49db0a89f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:22.208769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:22.208769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-245b2cf2-238f-42f0-9da0-10b62ee30a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:22.208769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 900/1787] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:21 2022] DELETE /identity/v3/projects/1f5f007a93fb4462b1a8ee49db0a89f6 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:22.212187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db187fc6-f5b7-4a82-aa81-e5886b0da30c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:22.212727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db187fc6-f5b7-4a82-aa81-e5886b0da30c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:22.214442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db187fc6-f5b7-4a82-aa81-e5886b0da30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:22.307074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-db187fc6-f5b7-4a82-aa81-e5886b0da30c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:22.308469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db187fc6-f5b7-4a82-aa81-e5886b0da30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:22.359760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 901/1788] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:22.370375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:22.382567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:22.382567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:22.432177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:22.436427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:22.446514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 902/1789] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:22.450001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:22.499835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a8b78-419f-46dc-9b8a-89c9b37e478e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:22.521931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:22.522709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:22.523176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:22.523663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:22.524375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:22.525449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:22.527400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:22.556723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 903/1790] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:02:22.561795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:22.578979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa95c5-8f99-40f2-bf6b-a51da9e77fbe admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:22.594430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:22.595037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:22.595365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:22.595668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:22.596901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:22.598453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:22.703028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 888/1791] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:21 2022] POST /identity/v3/projects => generated 369 bytes in 710 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:22.703583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:22.705534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:22.781113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 904/1792] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:22 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:22.797759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:22.835640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:22.852236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dac925-9ebf-4e2d-a943-52e931fa4a0f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.090608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db864a7d-0f56-407c-b3e7-662c3ce33c7e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.110999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.111981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.112510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.113032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.114039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.116648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.123143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.123517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.123982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.124458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.126242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.142178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:23.145462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405e528-058d-4413-a170-840fb4a9fc56 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.291994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 889/1793] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:22 2022] POST /identity/v3/users => generated 285 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:23.293513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.293815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.294104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.294579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 905/1794] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:22 2022] POST /identity/v3/users => generated 290 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:23.296526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.359150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb45c8-9c18-4ccc-94f4-93d69fa31d27 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.382417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] MFA Rules not processed for user `369d649af705470081765c82affb448d`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:23.384120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:23.385838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.386517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.386799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.387074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.395703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.408653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.434448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 906/1795] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:23.437342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.485153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c214e8-9c80-464d-a010-191fe71b42a4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.501071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.503209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.503495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.503777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.504496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.509885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.553553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 907/1796] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:23.554685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.572379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 890/1797] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:23.573368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.584078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6762d-8704-44b6-b036-71c5b331e73a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.594692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2b4e8-8437-41e5-8be1-dd4d2097f247 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.600990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.601588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.601878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.602163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.603160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.605657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.607952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.610170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.610938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.618928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.618928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.618928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.631585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 891/1798] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:23.633154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.635752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:23.644120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 908/1799] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:23.650422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.660682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a4fb4-4a01-4f0e-a63e-6d57747799d0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.667613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.668575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.669308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.669790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] PATH_INFO: `/v3/projects/ba74db68e81c4e419fc2395728bcaf34/users/877ed8a6c1074d46b3cedeec918f4273/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.676804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332f0d1-26d0-480e-9f94-6220699b814b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.684998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.685639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.685985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.686271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.686994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.688990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d61815-80a7-4c4e-a88e-8f598946ec95 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.690877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba74db68e81c4e419fc2395728bcaf34, user_id=877ed8a6c1074d46b3cedeec918f4273, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:23.693716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:23.792558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 909/1800] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:23 2022] PUT /identity/v3/projects/ba74db68e81c4e419fc2395728bcaf34/users/877ed8a6c1074d46b3cedeec918f4273/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:23.794158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:23.819165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 892/1801] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:23 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:23.820184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.820512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.821082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.874150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] MFA Rules not processed for user `877ed8a6c1074d46b3cedeec918f4273`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:23.875583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:23.890931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60704b97-9049-4937-8820-1bb53b96c206 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:23.973706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:23.974449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:23.974783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:23.975212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] PATH_INFO: `/v3/projects/cc59a0cd74974eceaf51ef36ba030316/users/419840dafffe4d7e85e8392d26734aee/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:23.985948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 893/1802] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:23.987086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.001471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793c06-c105-4a84-9c51-532795aa2c09 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.002855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc59a0cd74974eceaf51ef36ba030316, user_id=419840dafffe4d7e85e8392d26734aee, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.005705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.008098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.009073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.009643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.010141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.010964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.013390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7633d3-169e-422d-9001-09ec3023a750 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.092631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 910/1803] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:23 2022] PUT /identity/v3/projects/cc59a0cd74974eceaf51ef36ba030316/users/419840dafffe4d7e85e8392d26734aee/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:24.096901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.139323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 894/1804] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:23 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:24.141196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.141494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.142198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.164691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2803-6e8d-4967-9d4b-4f1d9b4407e1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.174505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.175141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.175422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.175692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.175889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] MFA Rules not processed for user `419840dafffe4d7e85e8392d26734aee`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:24.176447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.180212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:24.180352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.214761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:24.251585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 895/1805] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:24.252602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.269127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87bc82-fac0-4926-af17-d7f40a907fea None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.278243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 911/1806] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:24 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:24.278652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.278772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.278904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.278904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.278904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.278904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.279560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.290783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 896/1807] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:24.291728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.302424 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce8d4-df61-4fb3-a93a-198e4ad4fe58 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.362054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a0a8e-5783-4718-a031-a0fcc7169841 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.374525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.374525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.374525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.374525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.374949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.376776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.384820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.385550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.385933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.386288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.387067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.389763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.487956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 912/1808] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:24 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:24.488904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.509626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 897/1809] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:24 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:24.512684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.525518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674e22-0394-4fe9-924c-76b0e7f3e8ce admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.581312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ea72-0984-49a7-8525-93e1325684a9 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.583848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.584552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.584843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.585127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] PATH_INFO: `/v3/projects/949a2991488d46a88f4e86c39a4a78e2/users/7d753232b0844b6fb4ec8528a267c3c4/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.600050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949a2991488d46a88f4e86c39a4a78e2, user_id=7d753232b0844b6fb4ec8528a267c3c4, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.603401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.608901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.608901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.608901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.608901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.609312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.610375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.695011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 913/1810] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:24 2022] PUT /identity/v3/projects/949a2991488d46a88f4e86c39a4a78e2/users/7d753232b0844b6fb4ec8528a267c3c4/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:24.695859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.716305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 898/1811] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:24 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:24.717275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.731048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59ff7e-0b94-434d-a8da-94340a980e04 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.768374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675d5d0-5fff-412a-9768-0bbb8c5dac57 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.773493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.774119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.774925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.775317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.776184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.777801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.791850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 914/1812] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:24.791850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.792951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.793590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.793882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.794193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.794896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.796623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.801347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f828348-2493-4d6e-80cb-591d4f9074d2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.806576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.807145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.813045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 899/1813] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:24.813174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.813174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.813356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.813446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.813446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.818192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79866be-6817-4308-ad16-060275258e3d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.824624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 915/1814] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:24.824624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.824873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.824873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.824873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.824873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.825525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.827167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.836487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:24.839348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 900/1815] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:24.840188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.842835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736a22-749a-4bcb-961a-2220ff98ad49 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.848031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.848711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.848998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.849285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] PATH_INFO: `/v3/projects/949a2991488d46a88f4e86c39a4a78e2/users/7d753232b0844b6fb4ec8528a267c3c4/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.849896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948f262-9246-4b38-a4b9-f86b68b45064 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.851454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949a2991488d46a88f4e86c39a4a78e2, user_id=7d753232b0844b6fb4ec8528a267c3c4, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.855331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.858731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.858731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.858731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.858731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] PATH_INFO: `/v3/projects/03cc40902252498c8da7a3bbe9b6c519/users/665df46aad5a4089b81bee30214196e7/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.867175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 916/1816] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:24 2022] PUT /identity/v3/projects/949a2991488d46a88f4e86c39a4a78e2/users/7d753232b0844b6fb4ec8528a267c3c4/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:24.868262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:24.872721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cc40902252498c8da7a3bbe9b6c519, user_id=665df46aad5a4089b81bee30214196e7, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.876355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.932452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d327e-8850-45a1-aa1a-0d9bbb800207 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:24.937139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:24.937813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:24.938097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:24.938405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] PATH_INFO: `/v3/projects/a98395c1e880423085e5cb887d2a411c/users/47d2839cf81a4e6b981c5a8ab359a4d0/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:24.952556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98395c1e880423085e5cb887d2a411c, user_id=47d2839cf81a4e6b981c5a8ab359a4d0, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:24.955400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f8b71-c846-4a8b-9710-8dcc5345490b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:24.989934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 901/1817] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:24 2022] PUT /identity/v3/projects/03cc40902252498c8da7a3bbe9b6c519/users/665df46aad5a4089b81bee30214196e7/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:24.994205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.063712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffd92-3b9e-48dd-a047-31decd2d1260 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.069751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.070443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.070744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.071027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.071808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.073479 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.084551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 902/1818] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:25.090039 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.096600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 917/1819] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:24 2022] PUT /identity/v3/projects/a98395c1e880423085e5cb887d2a411c/users/47d2839cf81a4e6b981c5a8ab359a4d0/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:25.108725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.109012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.109237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd07c0-145a-4040-8a40-99071c8c2539 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.109353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.143448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] MFA Rules not processed for user `47d2839cf81a4e6b981c5a8ab359a4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:25.144964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:25.189490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.190283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.190613 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.190963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] PATH_INFO: `/v3/projects/03cc40902252498c8da7a3bbe9b6c519/users/665df46aad5a4089b81bee30214196e7/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.193887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cc40902252498c8da7a3bbe9b6c519, user_id=665df46aad5a4089b81bee30214196e7, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.198077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f16f2c-d93a-4005-8df4-abb835d06e49 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.213220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 903/1820] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:25 2022] PUT /identity/v3/projects/03cc40902252498c8da7a3bbe9b6c519/users/665df46aad5a4089b81bee30214196e7/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:25.224233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.224555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.225238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.251993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 918/1821] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:25.256836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.277189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991051c7-5de9-4154-958d-9e1776850d3c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.293368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] MFA Rules not processed for user `665df46aad5a4089b81bee30214196e7`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:25.299273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:25.324879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:25.342104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.342891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.343284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.343671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.344657 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.346841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.417495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 904/1822] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:25.430333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.435631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:25.442807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 919/1823] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:25 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:25.454460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.521020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9337d-5b15-4471-86e2-20c23b798dbd None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.558977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58180738-e034-4c1e-b3f4-d091f8327137 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.563656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.564288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.564635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.564919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.565593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.567624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.588499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.589163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.589453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.589738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.641637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 920/1824] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:25 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:25.645207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.656800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d8020-3fdf-4d1b-98ee-5d7178bb65fe admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.660439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:25.662098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.663981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.664276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.664958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.665254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.665541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.666373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.668697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.676833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 905/1825] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:02:25.683409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 921/1826] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:25.689082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.699107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fbdd-aa2b-432b-a11c-f3055309498b service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.704025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.704716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.705008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.705291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] PATH_INFO: `/v3/projects/be72663c85994cbeb7b3ec211b72e7c4/users/e7bd9647e66b49349b4c611c1f7ea080/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.720831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be72663c85994cbeb7b3ec211b72e7c4, user_id=e7bd9647e66b49349b4c611c1f7ea080, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.725177 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.801233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.803018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:25.816742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce9e-0688-4c5c-b773-c8f25d24277e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.820475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.821099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.822471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.822761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.850493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 906/1827] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:25 2022] PUT /identity/v3/projects/be72663c85994cbeb7b3ec211b72e7c4/users/e7bd9647e66b49349b4c611c1f7ea080/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:25.852292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.894818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.896379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.905614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 922/1828] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:02:25.921714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2076fe4-2c3d-4858-8b63-11ef014d4794 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.926423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.927135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.927529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.927755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.928476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.930102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d704b99-e8ae-4d10-8400-213fb9d32a2f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.941977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 907/1829] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:25.946202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:25.957229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30f12-ba79-4ac1-842c-2dda0cb8a75f service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:25.962151 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:25.962901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.963196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.967053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] PATH_INFO: `/v3/projects/be72663c85994cbeb7b3ec211b72e7c4/users/e7bd9647e66b49349b4c611c1f7ea080/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:25.967053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=be72663c85994cbeb7b3ec211b72e7c4, user_id=e7bd9647e66b49349b4c611c1f7ea080, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:25.968328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:25.979406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 923/1830] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:25 2022] PUT /identity/v3/projects/be72663c85994cbeb7b3ec211b72e7c4/users/e7bd9647e66b49349b4c611c1f7ea080/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:25.996073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:25.996938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:25.997408 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:26.035533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] MFA Rules not processed for user `e7bd9647e66b49349b4c611c1f7ea080`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:26.037131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:26.113153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 908/1831] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:26.126924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:26.187398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105554-2a56-4f9a-a964-cd5d4eb364ae admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:26.191072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:26.191692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:26.191975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:26.192252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:26.204638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:26.206635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cfad0-4dc3-4fe3-a97a-3069c255130b service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:26.221353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 924/1832] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:26.292678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:26.352699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:26.365997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd71b54-3086-40ad-8bdb-b49fa2bb5e1a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:26.373407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:26.374329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:26.374672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:26.375016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:26.475025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:26.477214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:26.499169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 909/1833] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:26.781981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:26.782532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:26.783028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:26.847997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] MFA Rules not processed for user `7d753232b0844b6fb4ec8528a267c3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:26.850126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:27.004144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 925/1834] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:27.012370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:27.185659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:27.650685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:27.650685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9ae79-c874-47d0-b99e-12c3eff95cf0 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:27.650685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:27.650685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:27.651398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:27.651398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:27.789856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:27.796124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:27.818324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 910/1835] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:28.283764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:28.617634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:29.832537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:30.263139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:30.823819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:31.669292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:31.759043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:31.770846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03e99c-93cd-4a2b-9aeb-84dfe87637c1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:31.779807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:31.779807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:31.779807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:31.779807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:31.780476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:31.847102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:31.848597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:31.859826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 926/1836] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:32.788823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:33.042061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:33.110117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48030c-2985-4a2c-ad4c-d0ab7f65fc8c service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:33.114928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:33.115580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:33.115866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:33.116167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] PATH_INFO: `/v3/users/419840dafffe4d7e85e8392d26734aee` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:33.117368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=419840dafffe4d7e85e8392d26734aee)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:33.119951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:33.283850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:33.297675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7123ae-1903-4d20-99c2-a2c008384044 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:33.302062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:33.302957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:33.303309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:33.303661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:33.314664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:33.316438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e41694-f55e-4e93-848d-81ea1b447f96 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:33.327727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 927/1837] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:02:33.359082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419840dafffe4d7e85e8392d26734aee'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:33.359881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419840dafffe4d7e85e8392d26734aee'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:33.360775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419840dafffe4d7e85e8392d26734aee'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:33.375187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419840dafffe4d7e85e8392d26734aee'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:33.451309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419840dafffe4d7e85e8392d26734aee'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:33.453000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:33.467986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 911/1838] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:33 2022] DELETE /identity/v3/users/419840dafffe4d7e85e8392d26734aee => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:33.472792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:33.473454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:33.473559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:33.493819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:33.495330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:33.557168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 928/1839] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:33.566540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:33.568199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:33.632937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4868e6-e075-4abc-ac40-006ee32b8d9c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:33.636651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:33.637301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:33.637590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:33.637866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:33.649934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:33.651443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:33.661301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 912/1840] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:34.235355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:34.236949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:34.250550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10afe9-72d3-4d70-8f64-263997051b7a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:34.255833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:34.256623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:34.256910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:34.257212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] PATH_INFO: `/v3/projects/cc59a0cd74974eceaf51ef36ba030316` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:34.259030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc59a0cd74974eceaf51ef36ba030316)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:34.261595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:34.379613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc59a0cd74974eceaf51ef36ba030316', 'request_id': 'req-55528bcb-f5ef-48fa-910a-66cc8e18221b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.386212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc59a0cd74974eceaf51ef36ba030316', 'request_id': 'req-55528bcb-f5ef-48fa-910a-66cc8e18221b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.392461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc59a0cd74974eceaf51ef36ba030316', 'request_id': 'req-55528bcb-f5ef-48fa-910a-66cc8e18221b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.473939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] The token cache is being invalidate because project cc59a0cd74974eceaf51ef36ba030316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:34.477484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.477484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.477484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 929/1841] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:34 2022] DELETE /identity/v3/projects/cc59a0cd74974eceaf51ef36ba030316 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:34.481196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:34.491013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab78aa-b671-4e29-8b87-0c0412252a3e service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:34.551689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:34.552336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:34.552642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:34.552913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] PATH_INFO: `/v3/users/665df46aad5a4089b81bee30214196e7` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:34.554516 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] RBAC: Authorizing `identity:delete_user(user_id=665df46aad5a4089b81bee30214196e7)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:34.557302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:34.766053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665df46aad5a4089b81bee30214196e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.766815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665df46aad5a4089b81bee30214196e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.767611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665df46aad5a4089b81bee30214196e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.781278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665df46aad5a4089b81bee30214196e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.862837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef70a249-a9fc-4d17-bc0c-fc292319366e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665df46aad5a4089b81bee30214196e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:34.877084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 913/1842] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:34 2022] DELETE /identity/v3/users/665df46aad5a4089b81bee30214196e7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:35.364373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:35.365944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:35.389080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55528bcb-f5ef-48fa-910a-66cc8e18221b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:35.435301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:35.450892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:35.451546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:35.451840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:35.452126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] PATH_INFO: `/v3/projects/03cc40902252498c8da7a3bbe9b6c519` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:35.453296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cc40902252498c8da7a3bbe9b6c519)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:35.455280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:35.590580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cc40902252498c8da7a3bbe9b6c519', 'request_id': 'req-d79dc396-fd6c-46e9-bd7b-0d77addb9157'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:35.596919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cc40902252498c8da7a3bbe9b6c519', 'request_id': 'req-d79dc396-fd6c-46e9-bd7b-0d77addb9157'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:35.603439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cc40902252498c8da7a3bbe9b6c519', 'request_id': 'req-d79dc396-fd6c-46e9-bd7b-0d77addb9157'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:35.728784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] The token cache is being invalidate because project 03cc40902252498c8da7a3bbe9b6c519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:35.729083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:35.729724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d79dc396-fd6c-46e9-bd7b-0d77addb9157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:35.731360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 930/1843] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:35 2022] DELETE /identity/v3/projects/03cc40902252498c8da7a3bbe9b6c519 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:35.736208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:35.736663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:35.736911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:35.757330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:35.758607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:35.763863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:35.819986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 914/1844] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:35.829510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:35.829804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:35.830086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:35.851029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:35.852770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:35.861326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 931/1845] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:35.872586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:35.882262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f239550-e1d8-498d-b477-13f925670ee5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:35.887149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:35.887701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:35.888210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:35.888664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:35.889751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:35.890066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:35.891716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:35.903535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 915/1846] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:35.907908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:35.917937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46890-c165-4355-8f21-d19c79e62fdd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:35.922771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:35.923382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:35.923787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:35.924072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:35.924741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:35.926751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.004813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 932/1847] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:36.014467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.025431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42822a37-f149-475b-b2e2-4b59301df46c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.083586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.084130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.084415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.084693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.085377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.088125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.185210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 916/1848] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:36 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:36.189383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.204300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa82f5-37c0-499f-bc0d-5d58e795c224 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.209245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.209873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.210385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.210505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.211752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.214007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.225421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 933/1849] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:36.230739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.241195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196019fc-ef5c-42d4-903d-9f45722ce05c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.250123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.250819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.251109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.251395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.252329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.254031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275793-1e53-49a0-9671-c44ac1c9400c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.269820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 917/1850] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:36.271841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.283275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c73c69-4b50-4c71-9aea-559b6c5209d3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.294827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.294827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.294827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.294827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] PATH_INFO: `/v3/projects/e0f27331ae1d469b9a9130faf1fa345f/users/b20a5a035b6a40458b4cc3c0b6ab87d7/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.309375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f27331ae1d469b9a9130faf1fa345f, user_id=b20a5a035b6a40458b4cc3c0b6ab87d7, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.312291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.378003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 934/1851] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:36 2022] PUT /identity/v3/projects/e0f27331ae1d469b9a9130faf1fa345f/users/b20a5a035b6a40458b4cc3c0b6ab87d7/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:36.389251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.389551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.389836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.419590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] MFA Rules not processed for user `b20a5a035b6a40458b4cc3c0b6ab87d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:36.420858 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:36.504954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 918/1852] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:36.509747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.578467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7eed7-9c51-4b4c-95f0-cd67fa07c8db admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.583523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.584146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.584430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.584706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.585370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.587857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.604768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:36.693125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 935/1853] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:36 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:36.698056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.709018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1180b-30a1-4686-b712-ed039bf32d6d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.713503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:36.769505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.770274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.770583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.770868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.771543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.773513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.902222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 919/1854] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:36 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:36.911495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.923329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f496ea-5866-40fb-81d6-2e0cd160d0f4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.928254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.928925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.929212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.929516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.930389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.932039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:36.945018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 936/1855] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:36.949612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:36.959860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7267f2-4a03-4bc6-8557-50a3d0df2013 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:36.965297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:36.965958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:36.966282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:36.966595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] PATH_INFO: `/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba/users/5333fdd3afcc4ae0ae889b872bbaaab1/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:36.982367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b30bc9cdfa4fd19983d5d381f7dfba, user_id=5333fdd3afcc4ae0ae889b872bbaaab1, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:36.986014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:37.069069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 920/1856] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:36 2022] PUT /identity/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba/users/5333fdd3afcc4ae0ae889b872bbaaab1/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:37.073187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:37.083507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333642-58bc-45cb-89d8-9863a0f57777 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:37.146500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:37.147239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:37.147620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:37.147994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:37.149555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:37.151229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659b5b-5b64-45e0-ab97-d8763953f7f3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:37.163883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 937/1857] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:37.167814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:37.181369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464827c-b098-4dbc-9f35-af9d34bff18e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:37.190940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:37.190940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:37.190940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:37.190940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] PATH_INFO: `/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba/users/5333fdd3afcc4ae0ae889b872bbaaab1/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:37.191325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b30bc9cdfa4fd19983d5d381f7dfba, user_id=5333fdd3afcc4ae0ae889b872bbaaab1, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:37.193076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:37.206573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 921/1858] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:37 2022] PUT /identity/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba/users/5333fdd3afcc4ae0ae889b872bbaaab1/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:37.217448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:37.217988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:37.219545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:37.253959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] MFA Rules not processed for user `5333fdd3afcc4ae0ae889b872bbaaab1`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:37.255343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:37.377978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 938/1859] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:37.389794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:37.391508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:37.522661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3604c8-b848-4bf8-8157-42b94d3e2170 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:37.527519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:37.529562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:37.529562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:37.529562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:37.542394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:37.543889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf22859-dbd6-4ba7-be42-48b1deebbd9b service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:37.555902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 922/1860] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:37.588559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:37.666573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fdb6a-a0b0-4a4f-b9a9-0fef022ccc3b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:37.671370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:37.671970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:37.672255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:37.672536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] PATH_INFO: `/v3/users/b20a5a035b6a40458b4cc3c0b6ab87d7` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:37.673714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20a5a035b6a40458b4cc3c0b6ab87d7)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:37.675682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:37.682784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:37.877150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20a5a035b6a40458b4cc3c0b6ab87d7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:37.877828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20a5a035b6a40458b4cc3c0b6ab87d7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:37.977042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20a5a035b6a40458b4cc3c0b6ab87d7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:37.992825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20a5a035b6a40458b4cc3c0b6ab87d7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:37.994198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20a5a035b6a40458b4cc3c0b6ab87d7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.008601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 939/1861] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:37 2022] DELETE /identity/v3/users/b20a5a035b6a40458b4cc3c0b6ab87d7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:38.013115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.013571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.013710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.033614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:38.034946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.129414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:38.131943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.147320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e82d0-4d36-450a-b57e-aa9a89716a56 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:38.149145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 923/1862] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:38.152527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:38.156171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.156325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:38.157754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.158804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.159312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] PATH_INFO: `/v3/users/877ed8a6c1074d46b3cedeec918f4273` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.161279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877ed8a6c1074d46b3cedeec918f4273)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:38.163143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:38.166621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:38.236075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e586f3-d09e-475a-b244-59e43b8bca98 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:38.239929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:38.240817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.241101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.241378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.252346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:38.253948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb13aa-8609-4b0a-acd1-8dfec5dc408a service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:38.265618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 924/1863] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:38.407036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877ed8a6c1074d46b3cedeec918f4273'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.407803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877ed8a6c1074d46b3cedeec918f4273'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.482663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877ed8a6c1074d46b3cedeec918f4273'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.504571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877ed8a6c1074d46b3cedeec918f4273'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.505459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fb7707e0-f4c5-4dae-b84b-27850c322c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877ed8a6c1074d46b3cedeec918f4273'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:38.522556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 940/1864] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:38 2022] DELETE /identity/v3/users/877ed8a6c1074d46b3cedeec918f4273 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:38.527593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.527987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.528333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.553154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:38.555993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.597807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118a7e-3f3d-46a1-9570-78b60c7e8610 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.599655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118a7e-3f3d-46a1-9570-78b60c7e8610 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.600031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1118a7e-3f3d-46a1-9570-78b60c7e8610 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.603515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 941/1865] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:02:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:02:38.614056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.614268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.614696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.675381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:38.675381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.687493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 925/1866] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:38.702097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:38.706457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.848430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:38.854743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38dad4-0e42-4154-a3c3-6070b52919b6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:38.861870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:38.862548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.862839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.863131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.885284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 942/1867] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:38.885284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:38.885284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:38.886387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:38.888702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:38.907507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8bf50-846e-48d7-b009-3b1a6f3829f9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:38.910442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 926/1868] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:38.920723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:38.921597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:38.922032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:38.922210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] PATH_INFO: `/v3/projects/e0f27331ae1d469b9a9130faf1fa345f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:38.923855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f27331ae1d469b9a9130faf1fa345f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:38.926421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:39.005807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:39.005807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:39.015719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ba8e1-7639-4b31-9440-c413166846fe service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:39.022049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:39.022817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:39.023175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:39.023569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:39.034862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:39.036567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:39.049445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 927/1869] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:02:39.207332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f27331ae1d469b9a9130faf1fa345f', 'request_id': 'req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.215648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f27331ae1d469b9a9130faf1fa345f', 'request_id': 'req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.222769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f27331ae1d469b9a9130faf1fa345f', 'request_id': 'req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.313726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] The token cache is being invalidate because project e0f27331ae1d469b9a9130faf1fa345f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:39.313726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.314316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.323771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 943/1870] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:38 2022] DELETE /identity/v3/projects/e0f27331ae1d469b9a9130faf1fa345f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:39.327990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:39.347581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ac99-ce09-46a5-bdcf-d1d4517e2cf6 service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:39.406783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:39.443730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:39.444396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:39.444741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:39.445156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] PATH_INFO: `/v3/users/5333fdd3afcc4ae0ae889b872bbaaab1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:39.446926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5333fdd3afcc4ae0ae889b872bbaaab1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:39.450390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:39.583043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:39.585523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:39.602752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e33d3-8a1a-4642-aea6-fedd964d3f29 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:39.608993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:39.609834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:39.610232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:39.610596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] PATH_INFO: `/v3/projects/ba74db68e81c4e419fc2395728bcaf34` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:39.611853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba74db68e81c4e419fc2395728bcaf34)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:39.614756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:39.693640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5333fdd3afcc4ae0ae889b872bbaaab1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.694308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5333fdd3afcc4ae0ae889b872bbaaab1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.694962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5333fdd3afcc4ae0ae889b872bbaaab1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.707721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5333fdd3afcc4ae0ae889b872bbaaab1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.745892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba74db68e81c4e419fc2395728bcaf34', 'request_id': 'req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.754012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba74db68e81c4e419fc2395728bcaf34', 'request_id': 'req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.761294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba74db68e81c4e419fc2395728bcaf34', 'request_id': 'req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.840374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5333fdd3afcc4ae0ae889b872bbaaab1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.854942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 928/1871] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:39 2022] DELETE /identity/v3/users/5333fdd3afcc4ae0ae889b872bbaaab1 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:39.897052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] The token cache is being invalidate because project ba74db68e81c4e419fc2395728bcaf34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:39.897356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.898009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:39.900013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 944/1872] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:39 2022] DELETE /identity/v3/projects/ba74db68e81c4e419fc2395728bcaf34 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:39.904103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:39.909803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:39.989540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b4a1f-f0ba-4e3b-a496-985bbacb1330 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:39.995775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:40.000280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.000280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.000280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] PATH_INFO: `/v3/users/47d2839cf81a4e6b981c5a8ab359a4d0` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.000280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d2839cf81a4e6b981c5a8ab359a4d0)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:40.000280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:40.261087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d2839cf81a4e6b981c5a8ab359a4d0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.261805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d2839cf81a4e6b981c5a8ab359a4d0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.262422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d2839cf81a4e6b981c5a8ab359a4d0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.271523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:40.279660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d2839cf81a4e6b981c5a8ab359a4d0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.379891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1476f1dd-a5eb-43c1-ba81-c6e109dc8d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d2839cf81a4e6b981c5a8ab359a4d0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.388026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:40.388798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:40.406096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 929/1873] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:39 2022] DELETE /identity/v3/users/47d2839cf81a4e6b981c5a8ab359a4d0 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:40.472813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f93c7-535c-4fde-9b2a-1e8e5815dbe7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:40.560041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:40.560707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.560997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.561275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] PATH_INFO: `/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.562596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b30bc9cdfa4fd19983d5d381f7dfba)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:40.564616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:40.704280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b30bc9cdfa4fd19983d5d381f7dfba', 'request_id': 'req-197f45ae-140b-482c-8341-55b1bdae4df3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.711649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b30bc9cdfa4fd19983d5d381f7dfba', 'request_id': 'req-197f45ae-140b-482c-8341-55b1bdae4df3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.717435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b30bc9cdfa4fd19983d5d381f7dfba', 'request_id': 'req-197f45ae-140b-482c-8341-55b1bdae4df3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.799628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] The token cache is being invalidate because project b0b30bc9cdfa4fd19983d5d381f7dfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:40.799958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.804552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:40.804552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 945/1874] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:40 2022] DELETE /identity/v3/projects/b0b30bc9cdfa4fd19983d5d381f7dfba => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:40.807336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be363-8187-49da-b7bd-63cc0fc82d4b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.807655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be363-8187-49da-b7bd-63cc0fc82d4b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.807935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6be363-8187-49da-b7bd-63cc0fc82d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.829227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9b6be363-8187-49da-b7bd-63cc0fc82d4b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:40.830561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9b6be363-8187-49da-b7bd-63cc0fc82d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:40.883321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:40.885711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:40.908427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 930/1875] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:40.921748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.921748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.921748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.939700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:40.941309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:40.950933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 931/1876] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:40.955210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:40.965130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2384552-d075-45e0-956b-f3f54a5a79ba None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:40.967760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f45ae-140b-482c-8341-55b1bdae4df3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:40.969910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:40.970553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.970837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.971113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.975001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:40.975135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:40.975214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:40.975214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:40.975345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:40.975345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:40.975345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:40.975624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] PATH_INFO: `/v3/projects/a98395c1e880423085e5cb887d2a411c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:40.976620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98395c1e880423085e5cb887d2a411c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:40.978585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:40.986629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 932/1877] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:02:40.991149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.001681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862934e7-1eea-4e71-b682-754c2c043892 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.006765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.007394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.007678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.008474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.009141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.011082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.100686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98395c1e880423085e5cb887d2a411c', 'request_id': 'req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.107760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98395c1e880423085e5cb887d2a411c', 'request_id': 'req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.113543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98395c1e880423085e5cb887d2a411c', 'request_id': 'req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.115267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 933/1878] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:41.120230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.183315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cd2c-fa93-4e37-90c3-76e19c1b6bb8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.188307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.188931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.189330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.189577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.190239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.192210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.207634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] The token cache is being invalidate because project a98395c1e880423085e5cb887d2a411c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:41.207930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.208598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.210577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 946/1879] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:40 2022] DELETE /identity/v3/projects/a98395c1e880423085e5cb887d2a411c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:41.215007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.216621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:41.267350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 934/1880] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:41 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:41.271652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.273781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:41.285934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb42a2-83ab-4ccc-965e-a5fd7c09a24f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.286804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da303-7ef3-4b17-b229-4d39e5572b6f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.291945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.293947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.294065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.294156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.294240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.294334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.294468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] PATH_INFO: `/v3/users/e7bd9647e66b49349b4c611c1f7ea080` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.294552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.295629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7bd9647e66b49349b4c611c1f7ea080)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.295747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.296521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.296706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.312091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 935/1881] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:41.313385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.325800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4ae5e-bf57-44bb-896d-d55f1167ae9f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.332320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.333050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.333343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.334178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.334396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.337717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.348954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 936/1882] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:41.357189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.363403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf3b2d-c181-4d84-8d90-d16d4c62d432 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.369359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.370325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.370641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.370925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] PATH_INFO: `/v3/projects/ac5a1fe0cc064dcc8514a2862820a177/users/fa30664cdb63469d8170d8e1ea34d1f3/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.394628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5a1fe0cc064dcc8514a2862820a177, user_id=fa30664cdb63469d8170d8e1ea34d1f3, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.398200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1b580-b172-425a-bf75-f28a144651cd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.477873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 937/1883] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:41 2022] PUT /identity/v3/projects/ac5a1fe0cc064dcc8514a2862820a177/users/fa30664cdb63469d8170d8e1ea34d1f3/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:41.479717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7bd9647e66b49349b4c611c1f7ea080'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.485049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7bd9647e66b49349b4c611c1f7ea080'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.489227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.489621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.489852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.525953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] MFA Rules not processed for user `fa30664cdb63469d8170d8e1ea34d1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:41.525953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:41.591514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7bd9647e66b49349b4c611c1f7ea080'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.592422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 938/1884] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:41.597069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.605336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7bd9647e66b49349b4c611c1f7ea080'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.606054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7bd9647e66b49349b4c611c1f7ea080'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:41.621731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 947/1885] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:41 2022] DELETE /identity/v3/users/e7bd9647e66b49349b4c611c1f7ea080 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:41.666147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db9c65-b99c-4310-a368-85d80482f22c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.720620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:41.734083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.734903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.735307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.735681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.736443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.738457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:41.831590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 939/1886] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:41.833596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:41.835717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:41.870103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0617-66ff-4ca4-be3c-056f708ad5a1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:41.962058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:41.962493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:41.962663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:41.962952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:41.963682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:41.966691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.068241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 948/1887] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:41 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:42.072429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.082632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287ecde-7d7b-474b-acde-ef749a638ced admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.085710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:42.088675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.089696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.089696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.089953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.090653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.094641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.095476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.097368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.111823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467df83-586e-497f-811c-b459db91d262 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.113476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 940/1888] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:42.118629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.119324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.119610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.120770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] PATH_INFO: `/v3/projects/be72663c85994cbeb7b3ec211b72e7c4` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.122668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.123312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be72663c85994cbeb7b3ec211b72e7c4)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.125487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.136601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121e02-7caf-4acc-92cf-c733e1ec78d0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.142571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.143432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.143736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.144020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] PATH_INFO: `/v3/projects/7f3dbe95091b4231a9d9bee500fd5631/users/c13c2d1301304901963c59b0ed42cfd9/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.163383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3dbe95091b4231a9d9bee500fd5631, user_id=c13c2d1301304901963c59b0ed42cfd9, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.167157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.261994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 941/1889] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:42 2022] PUT /identity/v3/projects/7f3dbe95091b4231a9d9bee500fd5631/users/c13c2d1301304901963c59b0ed42cfd9/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:42.265945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.269976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be72663c85994cbeb7b3ec211b72e7c4', 'request_id': 'req-359c04a4-3038-485c-8b85-3e438b7aaf74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:42.278049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be72663c85994cbeb7b3ec211b72e7c4', 'request_id': 'req-359c04a4-3038-485c-8b85-3e438b7aaf74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:42.284099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be72663c85994cbeb7b3ec211b72e7c4', 'request_id': 'req-359c04a4-3038-485c-8b85-3e438b7aaf74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:42.332355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101713ca-f4f8-485c-b9b2-033858c39023 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.361841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] The token cache is being invalidate because project be72663c85994cbeb7b3ec211b72e7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:42.365312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:42.365312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-359c04a4-3038-485c-8b85-3e438b7aaf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:42.365312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 949/1890] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:42 2022] DELETE /identity/v3/projects/be72663c85994cbeb7b3ec211b72e7c4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:02:42.370853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5210ce-6946-4169-8ab8-109a4e5ea5e5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.371147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5210ce-6946-4169-8ab8-109a4e5ea5e5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.371435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5210ce-6946-4169-8ab8-109a4e5ea5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.390603 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.391325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.391611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.395857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.395961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-cc5210ce-6946-4169-8ab8-109a4e5ea5e5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:42.396066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.396155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cc5210ce-6946-4169-8ab8-109a4e5ea5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.396242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.403755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 950/1891] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:02:42.411683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.412038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.412302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.417084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 942/1892] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:02:42.419950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.421818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.432396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:42.433618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.434471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326861c-e078-49c5-8b91-3d906acc0205 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.443025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 951/1893] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:42.448782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.450879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.451577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.451903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.452287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] PATH_INFO: `/v3/projects/7f3dbe95091b4231a9d9bee500fd5631/users/c13c2d1301304901963c59b0ed42cfd9/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.454174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3dbe95091b4231a9d9bee500fd5631, user_id=c13c2d1301304901963c59b0ed42cfd9, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.457184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96c94b-4777-42f9-b9f7-712f474701af admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.458989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf143b-71e3-470b-b7c6-dab653e1a55c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.465775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.466605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.466993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.467411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.468224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:42.469099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.469590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 943/1894] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:42 2022] PUT /identity/v3/projects/7f3dbe95091b4231a9d9bee500fd5631/users/c13c2d1301304901963c59b0ed42cfd9/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:42.470551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.479930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.480212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.480502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.483947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 952/1895] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:42.488436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.506756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347824fd-e8c8-44b2-b53b-389ed44173f8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.519046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] MFA Rules not processed for user `c13c2d1301304901963c59b0ed42cfd9`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:42.520906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.604722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.605443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.605799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.606083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.607040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.609097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.634307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 944/1896] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:42.657149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.663868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:42.663868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:42.720461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 953/1897] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:42 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:42.724888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.731461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d747c49-e522-4320-b1f7-15b344b8df7c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.792225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0958-75e3-4ced-87ef-69b924e6419b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.795003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.795700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.796001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.796283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.797305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.797616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.797899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.798174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.798866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.800782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.864929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.866582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.891441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 945/1898] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:42.947337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 954/1899] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:42 2022] POST /identity/v3/users => generated 284 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:42.948523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:42.959809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f7a6a-fc2c-4352-9782-cf70b23cc8a6 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:42.966198 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:42.967030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:42.967267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:42.967546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:42.968251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:42.969859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:42.981711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 946/1900] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:42.991318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.006433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28762492-440c-4dec-b0ed-a806552dcfde admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.028564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.028927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.029280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.029870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.031410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.040035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5790b6-695b-436a-b671-e03c5b12801a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.053231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:43.099797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 955/1901] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:43.108051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.128930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46575-b232-4595-9e1c-124ca634bfcd admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.138110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.139072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.143885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.143885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] PATH_INFO: `/v3/projects/f82bfb8d49684a9fb3c442bbbafdf825/users/7159f6b49c95407c8eb4b6196f79a0ad/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.162845 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82bfb8d49684a9fb3c442bbbafdf825, user_id=7159f6b49c95407c8eb4b6196f79a0ad, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.169144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.251251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 947/1902] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:43 2022] PUT /identity/v3/projects/f82bfb8d49684a9fb3c442bbbafdf825/users/7159f6b49c95407c8eb4b6196f79a0ad/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:43.264541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.264824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.265107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.294203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:43.295276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] MFA Rules not processed for user `7159f6b49c95407c8eb4b6196f79a0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:43.296545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:43.376283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 956/1903] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:43.386266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.450784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b42f9-adb9-4412-adee-a2bcba351259 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.455540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.456161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.456449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.456731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.457413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.459454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.551920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 948/1904] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:43 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:43.553406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.565950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffa349-ffe7-47d2-afeb-28c1cb8de85b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.625452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.626095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.626409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.626695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.627413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.629382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.731081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 957/1905] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:43 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:43.733814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.750397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253d77b-2911-46a3-81ff-a95293469779 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.754313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:43.762469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.763154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.763583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.764010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.764733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.768491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.798120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 949/1906] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:43.801026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:43.817543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9de0a-d362-4890-b31f-00cf73d1bd63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:43.844301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:43.846744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:43.847193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:43.847806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] PATH_INFO: `/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2/users/988de06163c344c79e37e9a5fc27fc67/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:43.881204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44916a2ee2314a7b9ccc768bdebb3ee2, user_id=988de06163c344c79e37e9a5fc27fc67, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:43.888762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:43.890091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:44.015699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 958/1907] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:43 2022] PUT /identity/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2/users/988de06163c344c79e37e9a5fc27fc67/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:44.019768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:44.035881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccb3e4-3ee6-4219-aa83-7a4ccac9a126 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:44.091579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:44.092215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:44.092496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:44.092772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:44.093466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:44.095110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dc5f8-74ea-45de-8bb2-a2c92d0f0caa admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:44.105615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 950/1908] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:44.109850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:44.121277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfad83-6dca-4282-bdaf-78a4ce8429b1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:44.125908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:44.126760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:44.127044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:44.127321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] PATH_INFO: `/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2/users/988de06163c344c79e37e9a5fc27fc67/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:44.129240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=44916a2ee2314a7b9ccc768bdebb3ee2, user_id=988de06163c344c79e37e9a5fc27fc67, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:44.131921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:44.144616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 959/1909] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:44 2022] PUT /identity/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2/users/988de06163c344c79e37e9a5fc27fc67/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:44.155532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638301-7709-4809-a576-8f95974d2470 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:44.155861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638301-7709-4809-a576-8f95974d2470 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:44.156229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638301-7709-4809-a576-8f95974d2470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:44.187589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-18638301-7709-4809-a576-8f95974d2470 None None] MFA Rules not processed for user `988de06163c344c79e37e9a5fc27fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:44.188844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-18638301-7709-4809-a576-8f95974d2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:44.270153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 951/1910] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:44.357609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:44.415561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6239a-cd5c-43a4-b166-f9b2b9dbf1ee admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:44.419894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:44.420661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:44.420946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:44.421263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:44.431271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:44.432751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:44.442661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 960/1911] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:44.935037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18638301-7709-4809-a576-8f95974d2470 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:44.937632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-18638301-7709-4809-a576-8f95974d2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:44.945050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:44.950174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18638301-7709-4809-a576-8f95974d2470 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:44.953812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:44.954489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:44.954783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:44.955427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:45.020217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:45.021744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa9a94-f800-4f46-b494-a313bb491c0d service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:45.038524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 952/1912] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:45.088540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:45.212156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:45.283021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc974459-f9a1-4902-beb1-319e59cdf781 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:45.288207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:45.288791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:45.289075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:45.289610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] PATH_INFO: `/v3/users/fa30664cdb63469d8170d8e1ea34d1f3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:45.291033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa30664cdb63469d8170d8e1ea34d1f3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:45.293038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:45.530254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa30664cdb63469d8170d8e1ea34d1f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:45.530941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa30664cdb63469d8170d8e1ea34d1f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:45.589128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa30664cdb63469d8170d8e1ea34d1f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:45.605612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa30664cdb63469d8170d8e1ea34d1f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:45.606663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa30664cdb63469d8170d8e1ea34d1f3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:45.620976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 961/1913] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:45 2022] DELETE /identity/v3/users/fa30664cdb63469d8170d8e1ea34d1f3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:45.625725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:45.625977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:45.626250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:45.646924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:45.648365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:45.724993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 953/1914] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:45.735463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:45.736939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:45.811854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f43274-374f-4531-a706-2e21033c99e4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:45.816007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:45.816865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:45.817163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:45.817455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:45.844712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:45.853324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:45.858456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:45.874816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 962/1915] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:46.459210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:46.462179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:46.474309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795fa01-2c3b-4c58-9cbc-c400521f3695 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:46.480553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:46.481315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:46.481567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:46.485160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] PATH_INFO: `/v3/projects/ac5a1fe0cc064dcc8514a2862820a177` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:46.485160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5a1fe0cc064dcc8514a2862820a177)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:46.485160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:46.643359 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5a1fe0cc064dcc8514a2862820a177', 'request_id': 'req-a81b0f7b-e102-4e27-b84f-f6be931bbfce'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:46.653875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5a1fe0cc064dcc8514a2862820a177', 'request_id': 'req-a81b0f7b-e102-4e27-b84f-f6be931bbfce'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:46.748688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5a1fe0cc064dcc8514a2862820a177', 'request_id': 'req-a81b0f7b-e102-4e27-b84f-f6be931bbfce'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:46.761816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] The token cache is being invalidate because project ac5a1fe0cc064dcc8514a2862820a177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:46.762372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:46.763138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:46.765052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 954/1916] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:46 2022] DELETE /identity/v3/projects/ac5a1fe0cc064dcc8514a2862820a177 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:46.769084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:46.780287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e08a3a-da3b-483b-bf67-d805b21148ce service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:46.849721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:46.849864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:46.850147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:46.850455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] PATH_INFO: `/v3/users/c13c2d1301304901963c59b0ed42cfd9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:46.852524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13c2d1301304901963c59b0ed42cfd9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:46.854521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:47.093808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13c2d1301304901963c59b0ed42cfd9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:47.094630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13c2d1301304901963c59b0ed42cfd9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:47.171073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13c2d1301304901963c59b0ed42cfd9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:47.184828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13c2d1301304901963c59b0ed42cfd9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:47.185540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1b6c81ad-f915-4501-8e45-db12ab9f2faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13c2d1301304901963c59b0ed42cfd9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:47.198340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 963/1917] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:02:46 2022] DELETE /identity/v3/users/c13c2d1301304901963c59b0ed42cfd9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:47.920970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:47.928751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:47.943356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b0f7b-e102-4e27-b84f-f6be931bbfce admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:47.968707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:48.002023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.002709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.002995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.003273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] PATH_INFO: `/v3/projects/7f3dbe95091b4231a9d9bee500fd5631` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.004436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3dbe95091b4231a9d9bee500fd5631)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.006396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.162725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3dbe95091b4231a9d9bee500fd5631', 'request_id': 'req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:48.169029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3dbe95091b4231a9d9bee500fd5631', 'request_id': 'req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:48.214791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3dbe95091b4231a9d9bee500fd5631', 'request_id': 'req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:48.228937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] The token cache is being invalidate because project 7f3dbe95091b4231a9d9bee500fd5631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:02:48.229236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:48.229915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-13f1b5cb-87ba-4bc7-98cb-b10c6c06ad74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:02:48.231486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 955/1918] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:02:47 2022] DELETE /identity/v3/projects/7f3dbe95091b4231a9d9bee500fd5631 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:48.237317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.237842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.237989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.260368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:48.261655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:48.275090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:48.324921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 964/1919] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:48.333243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.333536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.333824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.355164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:48.356369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:48.365068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 956/1920] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:48.369895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.379511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895151d-67bc-4056-98b9-6e25143c65dd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.384777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.385404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.385780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.386061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.386797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:02:48.387106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.388727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.401130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 965/1921] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:02:48.405815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.415382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a1fd-b4f6-4e13-8cb5-8e63e0a6a543 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.420203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.420815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.421099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.421381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.422059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.423997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.506688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 957/1922] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:02:48 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:02:48.511525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.521466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53329834-ac5d-44c9-9f5f-31d80396969c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.581590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.582253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.582576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.582859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.583587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.585529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.686315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 966/1923] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:02:48 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:02:48.690630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.702114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5b63-377f-4060-85d3-27d0ab3d899d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.707048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.707669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.707945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.711602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.711602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.711602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.722037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 958/1924] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:48.725927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.737430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f9360-6082-462b-acd1-d14a62ecff77 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.743008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.744337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.745061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.745320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.746040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.747757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db4ab-6b68-47ef-8fed-b904ab59c566 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.762146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 967/1925] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:02:48.768142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.778924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90fc26-786e-44a3-b144-a65360cb18d8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.783903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.784593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.784880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.785196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] PATH_INFO: `/v3/projects/015548578c01407490700514ec75c86f/users/019e016faa3b4953b4f1e253253fcb28/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.802734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015548578c01407490700514ec75c86f, user_id=019e016faa3b4953b4f1e253253fcb28, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.806235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.920906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 959/1926] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:02:48 2022] PUT /identity/v3/projects/015548578c01407490700514ec75c86f/users/019e016faa3b4953b4f1e253253fcb28/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:02:48.933956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.934574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.935328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.966582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] MFA Rules not processed for user `019e016faa3b4953b4f1e253253fcb28`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:02:48.967813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:49.035797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 968/1927] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:02:49.048791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:49.050585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:49.110561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed83ce-8161-456a-8535-377b36a6dad7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:49.114557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:49.115218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:49.115506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:49.115786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:49.123554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:49.125859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:49.127336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24035602-1aa7-4181-96aa-c238a8d8ada5 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:49.136703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 960/1928] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:02:59.032742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:59.221841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:06.333682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:06.335868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:06.408281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1373b3-a9ef-48b1-8659-b6ee0bfb34f3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:06.412939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:06.413590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:06.413879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:06.414222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] PATH_INFO: `/v3/users/7159f6b49c95407c8eb4b6196f79a0ad` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:06.415499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7159f6b49c95407c8eb4b6196f79a0ad)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:06.418167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:06.610886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7159f6b49c95407c8eb4b6196f79a0ad'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:06.611602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7159f6b49c95407c8eb4b6196f79a0ad'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:06.681223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7159f6b49c95407c8eb4b6196f79a0ad'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:06.695423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7159f6b49c95407c8eb4b6196f79a0ad'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:06.702140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7159f6b49c95407c8eb4b6196f79a0ad'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:06.709831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 969/1929] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:06 2022] DELETE /identity/v3/users/7159f6b49c95407c8eb4b6196f79a0ad => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:06.715257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3de1-e98b-4a38-80c0-14550cb28ab9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:06.715755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3de1-e98b-4a38-80c0-14550cb28ab9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:06.715848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d3de1-e98b-4a38-80c0-14550cb28ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:06.736883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-671d3de1-e98b-4a38-80c0-14550cb28ab9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:06.738107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-671d3de1-e98b-4a38-80c0-14550cb28ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:06.802055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 961/1930] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:06.812870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:06.815090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:06.878189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a375ca-3e95-4c71-bd95-581bcdeda15b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:06.882013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:06.882666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:06.882954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:06.883233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:06.892684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:06.894130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9172c-49a7-4ea8-873f-b77bfb3acab3 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:06.905784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 970/1931] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:06.938391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:06.938391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:06.938596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:06.940010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 962/1932] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:06.970391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24350f-f223-4a37-811a-206439799f18 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:06.970795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24350f-f223-4a37-811a-206439799f18 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:06.971138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24350f-f223-4a37-811a-206439799f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:06.997511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1e24350f-f223-4a37-811a-206439799f18 None None] MFA Rules not processed for user `90c8ff3469974d15a2a3ec8d7af9fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:06.998831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e24350f-f223-4a37-811a-206439799f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:07.016728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:07.062497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 971/1933] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:07.075698 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:07.078577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:07.098123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfcf1b-1bda-4d72-b2ce-29ef9ceff17a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:07.105109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:07.105785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:07.106101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:07.106442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:07.117163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:07.119295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:07.175693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 963/1934] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:07.523377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24350f-f223-4a37-811a-206439799f18 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:07.524894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e24350f-f223-4a37-811a-206439799f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:07.540271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24350f-f223-4a37-811a-206439799f18 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:07.545718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:07.546491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:07.546941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:07.547364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] PATH_INFO: `/v3/projects/f82bfb8d49684a9fb3c442bbbafdf825` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:07.548860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82bfb8d49684a9fb3c442bbbafdf825)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:07.550901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:07.675241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82bfb8d49684a9fb3c442bbbafdf825', 'request_id': 'req-40299869-9d0f-4823-8441-dc73abdf53ac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:07.683217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82bfb8d49684a9fb3c442bbbafdf825', 'request_id': 'req-40299869-9d0f-4823-8441-dc73abdf53ac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:07.690285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82bfb8d49684a9fb3c442bbbafdf825', 'request_id': 'req-40299869-9d0f-4823-8441-dc73abdf53ac'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:07.772339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] The token cache is being invalidate because project f82bfb8d49684a9fb3c442bbbafdf825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:07.772621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:07.773267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:07.774955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 972/1935] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:07 2022] DELETE /identity/v3/projects/f82bfb8d49684a9fb3c442bbbafdf825 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:07.779040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:07.790083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d92ad-491c-4541-b72c-f144e5e5ccdd service placement] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:07.859438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:07.860201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:07.860486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:07.860846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] PATH_INFO: `/v3/users/988de06163c344c79e37e9a5fc27fc67` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:07.862313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=988de06163c344c79e37e9a5fc27fc67)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:07.864397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:08.008628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:08.010228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:08.023691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40299869-9d0f-4823-8441-dc73abdf53ac admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:08.028571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:08.029216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:08.029497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:08.029796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] PATH_INFO: `/v3/users/369d649af705470081765c82affb448d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:08.030975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=369d649af705470081765c82affb448d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:08.032899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:08.056155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988de06163c344c79e37e9a5fc27fc67'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.057533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988de06163c344c79e37e9a5fc27fc67'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.058308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988de06163c344c79e37e9a5fc27fc67'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.070523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988de06163c344c79e37e9a5fc27fc67'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.144419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988de06163c344c79e37e9a5fc27fc67'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.157758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 964/1936] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:07 2022] DELETE /identity/v3/users/988de06163c344c79e37e9a5fc27fc67 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:08.239689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369d649af705470081765c82affb448d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.240372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369d649af705470081765c82affb448d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.330224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369d649af705470081765c82affb448d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.343837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369d649af705470081765c82affb448d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.344517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369d649af705470081765c82affb448d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.358017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 973/1937] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:08 2022] DELETE /identity/v3/users/369d649af705470081765c82affb448d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:08.658082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:08.658427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:08.659587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:08.661166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:08.671974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:08.678257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:08.733377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbf1e6-007e-49ef-9c72-cdda27283155 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:08.737474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e605ac-498a-432c-acf2-03cc0bc1b13d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:08.738173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:08.739060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:08.739349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:08.739635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] PATH_INFO: `/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:08.740805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44916a2ee2314a7b9ccc768bdebb3ee2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:08.742544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:08.742767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:08.743239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:08.743520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:08.748233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] PATH_INFO: `/v3/projects/6fcd512718fc4562b987412daea3767a` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:08.748233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fcd512718fc4562b987412daea3767a)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:08.748233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:08.864645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44916a2ee2314a7b9ccc768bdebb3ee2', 'request_id': 'req-26905cb4-2599-4500-ac21-381a111a9af5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.865052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fcd512718fc4562b987412daea3767a', 'request_id': 'req-6a4c1431-c6af-4618-9d1e-9c1a48b54777'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.871523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fcd512718fc4562b987412daea3767a', 'request_id': 'req-6a4c1431-c6af-4618-9d1e-9c1a48b54777'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.871791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44916a2ee2314a7b9ccc768bdebb3ee2', 'request_id': 'req-26905cb4-2599-4500-ac21-381a111a9af5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.878141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44916a2ee2314a7b9ccc768bdebb3ee2', 'request_id': 'req-26905cb4-2599-4500-ac21-381a111a9af5'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.958806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fcd512718fc4562b987412daea3767a', 'request_id': 'req-6a4c1431-c6af-4618-9d1e-9c1a48b54777'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.971686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] The token cache is being invalidate because project 6fcd512718fc4562b987412daea3767a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:08.972111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.972791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:08.974325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 965/1938] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:08 2022] DELETE /identity/v3/projects/6fcd512718fc4562b987412daea3767a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:08.978618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:08.980104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.050694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c1431-c6af-4618-9d1e-9c1a48b54777 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.051943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] The token cache is being invalidate because project 44916a2ee2314a7b9ccc768bdebb3ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:09.052235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.052869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-26905cb4-2599-4500-ac21-381a111a9af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.054528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 974/1939] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:08 2022] DELETE /identity/v3/projects/44916a2ee2314a7b9ccc768bdebb3ee2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:09.059945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2854-f7e5-4241-a424-f3f781b343dd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.060375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2854-f7e5-4241-a424-f3f781b343dd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.060718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2854-f7e5-4241-a424-f3f781b343dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.081000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-01bb2854-f7e5-4241-a424-f3f781b343dd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:09.082308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-01bb2854-f7e5-4241-a424-f3f781b343dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.095443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:09.110967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.111711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.111908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.112188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] PATH_INFO: `/v3/users/7d753232b0844b6fb4ec8528a267c3c4` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.113381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d753232b0844b6fb4ec8528a267c3c4)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.115443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.146159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 975/1940] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:09.155344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.155635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.155914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.175604 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:09.176769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.185543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 976/1941] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:09.189660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.199514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde267e0-05c7-4e09-9f1d-769a43719c24 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.204218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.204833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.205098 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.205380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.206076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:09.207319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.208870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.220821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 977/1942] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:09.225102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.234841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb7b3-cc30-4d3b-ac48-aa3372cb8a1a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.239479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.240099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.240388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.240669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.244852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.244852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.278564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d753232b0844b6fb4ec8528a267c3c4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.279342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d753232b0844b6fb4ec8528a267c3c4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.279970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d753232b0844b6fb4ec8528a267c3c4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.292972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d753232b0844b6fb4ec8528a267c3c4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.361069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 978/1943] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:09.362212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.374624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.374624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:09.376041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d753232b0844b6fb4ec8528a267c3c4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.392880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 966/1944] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:08 2022] DELETE /identity/v3/users/7d753232b0844b6fb4ec8528a267c3c4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:09.448770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12097413-0889-4412-aa8e-104e37be31ee admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.507386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.508044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.508327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.508606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.509271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.511480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.633988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 979/1945] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:09.638233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.648460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5950dda-8be2-444c-91d8-66d9310adcb2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.653403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.654015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.654308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.654620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.659707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.659707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.664280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.667229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.668092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 967/1946] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:09.671918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.681579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de1a66-5083-454e-bad9-88af7a342850 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.684087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698ffc-1fbb-4ada-8550-ab17e6cfc40e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.688995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.689652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.689940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.690221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.690955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.692665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.701056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.701719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.702009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.702293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] PATH_INFO: `/v3/projects/949a2991488d46a88f4e86c39a4a78e2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.703530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] RBAC: Authorizing `identity:delete_project(project_id=949a2991488d46a88f4e86c39a4a78e2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.707621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 968/1947] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:09.707774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.709095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.719824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95d2cc-54dd-4dd1-9545-c988f0ad0efd admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:09.724588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:09.725251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.725541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.725825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] PATH_INFO: `/v3/projects/a65f71f294514e4aac6ad8f958e9d500/users/81888ef33ce340c0864befec341384e1/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.744465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65f71f294514e4aac6ad8f958e9d500, user_id=81888ef33ce340c0864befec341384e1, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:09.747207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d0c8-4ecb-42b6-9dcb-fd25db671191 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:09.850521 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 969/1948] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:09 2022] PUT /identity/v3/projects/a65f71f294514e4aac6ad8f958e9d500/users/81888ef33ce340c0864befec341384e1/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:09.859735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949a2991488d46a88f4e86c39a4a78e2', 'request_id': 'req-8cc65972-851f-4d24-ae34-bd52537aefda'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.864649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.864649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.864649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.867394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949a2991488d46a88f4e86c39a4a78e2', 'request_id': 'req-8cc65972-851f-4d24-ae34-bd52537aefda'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.873571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949a2991488d46a88f4e86c39a4a78e2', 'request_id': 'req-8cc65972-851f-4d24-ae34-bd52537aefda'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.892156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] MFA Rules not processed for user `81888ef33ce340c0864befec341384e1`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:09.893476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:09.960154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 970/1949] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:09.960719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] The token cache is being invalidate because project 949a2991488d46a88f4e86c39a4a78e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:09.961014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.961705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8cc65972-851f-4d24-ae34-bd52537aefda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:09.963958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 980/1950] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:09 2022] DELETE /identity/v3/projects/949a2991488d46a88f4e86c39a4a78e2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:09.965148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:09.968476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dd3f-476b-4d85-bc23-ce24cd54a4a8 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:09.968775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dd3f-476b-4d85-bc23-ce24cd54a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:09.969188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2dd3f-476b-4d85-bc23-ce24cd54a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:09.973986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:09.995973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8da2dd3f-476b-4d85-bc23-ce24cd54a4a8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:09.997201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8da2dd3f-476b-4d85-bc23-ce24cd54a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:10.022631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:10.036848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de9ae0-2bb7-4f70-aad2-d0716b8255f0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.046542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.047178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.047464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.047748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.048427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.050741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.082095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 981/1951] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:10.092358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.092764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.093262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.113434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:10.115471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:10.124218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 982/1952] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:10.128424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.138542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcf79e-af8a-416c-b4f8-9ea4a2d8233c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.144599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.145356 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.145675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.145961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.146864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:10.147054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.148720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.161824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 983/1953] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:10.166553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.177148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeb683-8ed5-4484-83bc-46cd5a21f41a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.183897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.187318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 971/1954] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:09 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:10.187450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.187450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.187450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.187450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.189863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.190624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.191486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:10.255633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93716b-80c9-4b72-b468-edc29cc8b30f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.260880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.261506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.261795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.262076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.262773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.264718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.329414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 984/1955] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:10.333647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.368135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 972/1956] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:10.372675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.386800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48179470-17d7-4f38-a971-7b13f8a7bff4 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.411887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701efc4-fac4-4bb3-9c24-712a81c39c0c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.416988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.417616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.417904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.418191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.419898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.421902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.455935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.456571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.462509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.462509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.462509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.462509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.469812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 973/1957] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:10.474159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.488484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512088ea-6140-4f83-90d8-22635b22f5fe admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.495539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.496228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.496517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.496803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] PATH_INFO: `/v3/projects/8c2466d85c63413ea32b78cc4514ceb9/users/7adf9300197b4769b6d336a6f2b7f0d3/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.517158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2466d85c63413ea32b78cc4514ceb9, user_id=7adf9300197b4769b6d336a6f2b7f0d3, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.519926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.522188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:10.564012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 985/1958] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:10.565018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.576304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b0af5-a2f3-49f5-9f29-e13ca5e3e2fb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.581212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.581828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.582116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.582431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.583134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.585055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.595115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 986/1959] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:10.599083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.609815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc31d0-b1dd-4a41-9515-0322a1db37fe admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.616663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 974/1960] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:10 2022] PUT /identity/v3/projects/8c2466d85c63413ea32b78cc4514ceb9/users/7adf9300197b4769b6d336a6f2b7f0d3/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:10.617649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.670085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.670857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.671149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.671428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.672129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.673770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.675223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:10.682368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545dc7e-7dc8-4e85-af08-69d09c11e91d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.686377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 987/1961] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:10.689096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.690883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.694090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.694090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.694090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.694090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.694454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.700232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce95b-b34e-4a02-a9e8-7525b1a605a5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.704985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.705639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.709192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 975/1962] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:10.709310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.709310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] PATH_INFO: `/v3/projects/5301eca5d8744d5082b48e174d0beebb/users/1d93059de316481da38452ebd4c2e769/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.709889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:10.720129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18406a-6268-46f5-97f8-99b5bad5953e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:10.721620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5301eca5d8744d5082b48e174d0beebb, user_id=1d93059de316481da38452ebd4c2e769, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.724349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef40d-002c-4566-989b-e746d14b718c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.728577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:10.728577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.728577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.728577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] PATH_INFO: `/v3/projects/8c2466d85c63413ea32b78cc4514ceb9/users/7adf9300197b4769b6d336a6f2b7f0d3/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.729167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2466d85c63413ea32b78cc4514ceb9, user_id=7adf9300197b4769b6d336a6f2b7f0d3, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:10.731913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8df1d3-aaa4-49a7-b31b-70c24257e748 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:10.742357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 976/1963] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:10 2022] PUT /identity/v3/projects/8c2466d85c63413ea32b78cc4514ceb9/users/7adf9300197b4769b6d336a6f2b7f0d3/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:10.759544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.760435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.760668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.787947 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] MFA Rules not processed for user `7adf9300197b4769b6d336a6f2b7f0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:10.789186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:10.831992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 988/1964] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:10 2022] PUT /identity/v3/projects/5301eca5d8744d5082b48e174d0beebb/users/1d93059de316481da38452ebd4c2e769/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:10.833247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:10.833541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:10.833818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:10.870950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] MFA Rules not processed for user `1d93059de316481da38452ebd4c2e769`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:10.872194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:10.880054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 977/1965] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:10.943000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 989/1966] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:13.375635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:13.377078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:13.435454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c960881-d844-43ee-a9d1-19fd595ec201 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:13.449399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:13.487097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:13.487740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:13.488027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:13.488308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:13.552618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:13.554186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:13.565047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 978/1967] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:13.694728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:13.705422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53402c-9f54-489f-a5a0-6b99e97f7ba2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:13.709087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:13.709713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:13.710001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:13.710285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:13.718856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:13.720359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3fe9a-3003-482e-9093-9b6813b252ef service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:13.729517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 990/1968] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:13.749510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:13.820058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245bd3-1da3-495c-8e07-486c5f064a41 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:13.824961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:13.825619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:13.825929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:13.826216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] PATH_INFO: `/v3/users/1d93059de316481da38452ebd4c2e769` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:13.828103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d93059de316481da38452ebd4c2e769)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:13.830097 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:14.020403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d93059de316481da38452ebd4c2e769'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.021189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d93059de316481da38452ebd4c2e769'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.021808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d93059de316481da38452ebd4c2e769'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.047192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d93059de316481da38452ebd4c2e769'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.050986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:14.112387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d93059de316481da38452ebd4c2e769'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.125500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 979/1969] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:13 2022] DELETE /identity/v3/users/1d93059de316481da38452ebd4c2e769 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:14.133102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:14.133102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:14.133102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:14.152299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:14.158511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:14.224720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 991/1970] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:14.235466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:14.237338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:14.300844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac2e5-67d6-4a6e-a237-4094e7b22e73 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:14.304775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:14.305463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:14.305766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:14.306043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:14.316045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:14.317976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:14.329660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 980/1971] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:14.606375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:14.608077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:14.622495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d3c7f-42ae-4852-a319-3f2f681fe79c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:14.628111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:14.628689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:14.628999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:14.629281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] PATH_INFO: `/v3/users/81888ef33ce340c0864befec341384e1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:14.630617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81888ef33ce340c0864befec341384e1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:14.632630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:14.804512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:14.805972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:14.819203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee3a29-8587-4b5a-9ca6-1d044d32dc59 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:14.827860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:14.828533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:14.834936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:14.834936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] PATH_INFO: `/v3/projects/5301eca5d8744d5082b48e174d0beebb` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:14.834936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5301eca5d8744d5082b48e174d0beebb)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:14.834936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:14.927832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:14.931218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81888ef33ce340c0864befec341384e1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.933314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81888ef33ce340c0864befec341384e1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.958664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5301eca5d8744d5082b48e174d0beebb', 'request_id': 'req-77f5fa43-38cd-4ffb-a176-9ef05db855f5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:14.964700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5301eca5d8744d5082b48e174d0beebb', 'request_id': 'req-77f5fa43-38cd-4ffb-a176-9ef05db855f5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.032696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81888ef33ce340c0864befec341384e1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.045222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81888ef33ce340c0864befec341384e1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.045876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ac96a7a0-2599-4f81-947a-7e27b1617cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81888ef33ce340c0864befec341384e1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.058907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 992/1972] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:14 2022] DELETE /identity/v3/users/81888ef33ce340c0864befec341384e1 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:15.063138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637bd7-0b8a-495c-b264-e9d48170e096 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.063432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637bd7-0b8a-495c-b264-e9d48170e096 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.063716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637bd7-0b8a-495c-b264-e9d48170e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.084371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c9637bd7-0b8a-495c-b264-e9d48170e096 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:15.085593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c9637bd7-0b8a-495c-b264-e9d48170e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.121140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5301eca5d8744d5082b48e174d0beebb', 'request_id': 'req-77f5fa43-38cd-4ffb-a176-9ef05db855f5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.144684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] The token cache is being invalidate because project 5301eca5d8744d5082b48e174d0beebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:15.144985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.145745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.153394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 981/1973] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:14 2022] DELETE /identity/v3/projects/5301eca5d8744d5082b48e174d0beebb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:15.153394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.153394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.154665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 993/1974] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:15.156089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.156385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.156667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.176578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:15.178091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.187659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 994/1975] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:15.190493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.190493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.225293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5fa43-38cd-4ffb-a176-9ef05db855f5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.230370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.231040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.231328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.231606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] PATH_INFO: `/v3/users/f3518daa855a4fff8839b5094fa61514` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.232804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3518daa855a4fff8839b5094fa61514)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.237693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.248859 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d764d0-05b2-4183-90b8-c8b6d369a116 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.254916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.255542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.255824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.256275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.257546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:15.266330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.267849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffac6e0-0dfe-4c8f-a2ec-1aed2a510895 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.278059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 995/1976] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:15.279454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.279743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.280027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.300444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:15.301784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.310688 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 996/1977] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:15.314626 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.324956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f150f4c-dd28-4600-9829-c04c9ed5897d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.329906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.330549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.330834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.331110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.331808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:15.332125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.333711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.346209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 997/1978] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:15.351956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.375076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4337b-3376-432a-8841-1dab0ab475b2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.378732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.379661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.380158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.380839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.381513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.383619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.446566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3518daa855a4fff8839b5094fa61514'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.447367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3518daa855a4fff8839b5094fa61514'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.448025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3518daa855a4fff8839b5094fa61514'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.460052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3518daa855a4fff8839b5094fa61514'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.480458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 998/1979] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:15.484304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.486003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.549760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2c8d9e62-cb53-402b-b980-49e5112df693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3518daa855a4fff8839b5094fa61514'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.560106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52249de-aa02-4b86-99de-a45a6a0aa248 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.564555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 982/1980] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:15 2022] DELETE /identity/v3/users/f3518daa855a4fff8839b5094fa61514 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:15.565190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.565846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.566189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.566504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.567169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.569064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.569569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.570003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.570393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.590916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:15.592478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.655354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 983/1981] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:15.665009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.666987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.676106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 999/1982] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:15.680055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.691863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c414c-e62d-436e-aa3e-ed642a7c642d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.696555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.697280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.702584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.702584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.702584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.702584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.710515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1000/1983] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:15.714840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.724793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6668-02b8-4c3c-8e1f-b8e979f73bda admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.730532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1747b-156c-4599-a137-19b1291a0235 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.731345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.731598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.731945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.732183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.732886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.734530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.734702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.735340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.735637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.735903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.746462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.751382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.751539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1001/1984] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:15.752955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.760249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 984/1985] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:15.764725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48065d-82c8-4642-9a80-ecdffa343f50 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.769434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:15.770023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.770740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.771054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.771310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.771601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] PATH_INFO: `/v3/projects/90ae9c279a1e411b812920790a907b75/users/b2ea6888a8334b2b967ec6ce4668d06e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.785672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095a3a-f37c-463d-9dc4-29c97335bb1f service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:15.789916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ae9c279a1e411b812920790a907b75, user_id=b2ea6888a8334b2b967ec6ce4668d06e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.791426 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:15.792504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.792919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.793268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] PATH_INFO: `/v3/projects/a65f71f294514e4aac6ad8f958e9d500` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.794754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65f71f294514e4aac6ad8f958e9d500)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:15.795555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89398561-e3da-4352-a5d2-16e7fbba27dd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.797549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:15.881962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1002/1986] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:15 2022] PUT /identity/v3/projects/90ae9c279a1e411b812920790a907b75/users/b2ea6888a8334b2b967ec6ce4668d06e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:15.893576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:15.893869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:15.894150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:15.924379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65f71f294514e4aac6ad8f958e9d500', 'request_id': 'req-072e3883-341e-4ca7-8108-8333dbfe8f59'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:15.927576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] MFA Rules not processed for user `b2ea6888a8334b2b967ec6ce4668d06e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:15.929095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:15.931183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65f71f294514e4aac6ad8f958e9d500', 'request_id': 'req-072e3883-341e-4ca7-8108-8333dbfe8f59'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.007063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1003/1987] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:16.028101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.029512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.045223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65f71f294514e4aac6ad8f958e9d500', 'request_id': 'req-072e3883-341e-4ca7-8108-8333dbfe8f59'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.079632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] The token cache is being invalidate because project a65f71f294514e4aac6ad8f958e9d500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:16.079632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.079632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.079632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 985/1988] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:15 2022] DELETE /identity/v3/projects/a65f71f294514e4aac6ad8f958e9d500 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:16.082546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.084671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.119555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6d22e-7c43-44f0-bd7e-8bcaff2ab9be None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.125414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.126326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.126810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.127165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.131002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.154099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.154099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:16.157682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1004/1989] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:16.169123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e3883-341e-4ca7-8108-8333dbfe8f59 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.174196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.174889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.175171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.175455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] PATH_INFO: `/v3/users/7adf9300197b4769b6d336a6f2b7f0d3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.178416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7adf9300197b4769b6d336a6f2b7f0d3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.178761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:16.179431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.181039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.196490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f98fb8-98e4-447a-bb2e-ef244eabc6c1 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.202620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.203273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.203562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.203881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] PATH_INFO: `/v3/users/b2ea6888a8334b2b967ec6ce4668d06e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.205396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ea6888a8334b2b967ec6ce4668d06e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.207048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:16.377754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ea6888a8334b2b967ec6ce4668d06e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.378476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ea6888a8334b2b967ec6ce4668d06e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.398525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7adf9300197b4769b6d336a6f2b7f0d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.399263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7adf9300197b4769b6d336a6f2b7f0d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.400900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7adf9300197b4769b6d336a6f2b7f0d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.413458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7adf9300197b4769b6d336a6f2b7f0d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.450205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ea6888a8334b2b967ec6ce4668d06e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.463614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ea6888a8334b2b967ec6ce4668d06e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.464314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ea6888a8334b2b967ec6ce4668d06e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.479327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1005/1990] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:16 2022] DELETE /identity/v3/users/b2ea6888a8334b2b967ec6ce4668d06e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:16.480220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.481833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:16.483091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.506808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4300e9e-a43f-479a-a2e9-6ae0892144fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7adf9300197b4769b6d336a6f2b7f0d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.521713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 986/1991] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:16 2022] DELETE /identity/v3/users/7adf9300197b4769b6d336a6f2b7f0d3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:16.523255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.523547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.523829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.543226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:16.544453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.551297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b25c-62d7-48dc-a915-5a60bc95b625 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.607156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 987/1992] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:16.617663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.618646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.619545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.620149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] PATH_INFO: `/v3/projects/c480fcb2f0fe427d860c83be84737011` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.621305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c480fcb2f0fe427d860c83be84737011)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.623286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:16.625496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.626963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:16.685115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b47344-2f22-49d1-921c-09d5b10e7f8d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.688765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.689388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.689671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.690028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.700355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.701806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:16.711810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 988/1993] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:16.719012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:16.736918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c480fcb2f0fe427d860c83be84737011', 'request_id': 'req-17917b1e-8a59-422a-b1ea-82a300feb0bc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.743224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c480fcb2f0fe427d860c83be84737011', 'request_id': 'req-17917b1e-8a59-422a-b1ea-82a300feb0bc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.751189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c480fcb2f0fe427d860c83be84737011', 'request_id': 'req-17917b1e-8a59-422a-b1ea-82a300feb0bc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.848787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] The token cache is being invalidate because project c480fcb2f0fe427d860c83be84737011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:16.849269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.850268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:16.852455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1006/1994] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:16 2022] DELETE /identity/v3/projects/c480fcb2f0fe427d860c83be84737011 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:16.856836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:16.868157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1ecee-cd7b-4fc7-ac93-f8ffcd640f02 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:16.946286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:16.946962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:16.947588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:16.948236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] PATH_INFO: `/v3/users/e7d3ae232fa54bc29b28bee275f7a2ec` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:16.949598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d3ae232fa54bc29b28bee275f7a2ec)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:16.951616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:17.207480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:17.209109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:17.219971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d3ae232fa54bc29b28bee275f7a2ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.220612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d3ae232fa54bc29b28bee275f7a2ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.221746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d3ae232fa54bc29b28bee275f7a2ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.237108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917b1e-8a59-422a-b1ea-82a300feb0bc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:17.237279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d3ae232fa54bc29b28bee275f7a2ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.244383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:17.245203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.245500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.245904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] PATH_INFO: `/v3/projects/8c2466d85c63413ea32b78cc4514ceb9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.247382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2466d85c63413ea32b78cc4514ceb9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:17.249455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:17.355851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-69f10624-de00-4855-a747-0e26c90f4b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d3ae232fa54bc29b28bee275f7a2ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.373183 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 989/1995] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:16 2022] DELETE /identity/v3/users/e7d3ae232fa54bc29b28bee275f7a2ec => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:17.381684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2466d85c63413ea32b78cc4514ceb9', 'request_id': 'req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.391488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2466d85c63413ea32b78cc4514ceb9', 'request_id': 'req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.409705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2466d85c63413ea32b78cc4514ceb9', 'request_id': 'req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.527082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] The token cache is being invalidate because project 8c2466d85c63413ea32b78cc4514ceb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:17.527377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.528011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.529616 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1007/1996] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:17 2022] DELETE /identity/v3/projects/8c2466d85c63413ea32b78cc4514ceb9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:17.534986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097574-1f74-480e-89a6-0baf38179455 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.535302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097574-1f74-480e-89a6-0baf38179455 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.535583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097574-1f74-480e-89a6-0baf38179455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.555941 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-43097574-1f74-480e-89a6-0baf38179455 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:17.557876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-43097574-1f74-480e-89a6-0baf38179455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:17.567426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:17.569147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:17.619281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:17.648344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 990/1997] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:17.656905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c0f7-340b-40cb-bcfa-b8ab61f5233f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:17.657121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.657608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.658290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.661931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:17.662571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.662875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.663214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] PATH_INFO: `/v3/projects/90ae9c279a1e411b812920790a907b75` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.664365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ae9c279a1e411b812920790a907b75)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:17.666653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:17.682708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:17.684003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:17.694302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 991/1998] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:17.699822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:17.709375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38047728-d7ff-4f8a-b558-ad2c22775fce None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:17.714871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:17.715485 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.715769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.716048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.716752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:17.717059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:17.718691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:17.734216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 992/1999] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:17.735679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:17.746871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b064d3c-dffd-4d1d-bf95-c43bf954c21d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:17.751922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:17.752553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:17.752834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:17.753119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:17.753838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:17.755771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:17.794228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ae9c279a1e411b812920790a907b75', 'request_id': 'req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.800350 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ae9c279a1e411b812920790a907b75', 'request_id': 'req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.806158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ae9c279a1e411b812920790a907b75', 'request_id': 'req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:17.966773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 993/2000] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:17 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:17.966773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:17.967778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.017947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] The token cache is being invalidate because project 90ae9c279a1e411b812920790a907b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:18.019710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.019710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.024777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1008/2001] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:17 2022] DELETE /identity/v3/projects/90ae9c279a1e411b812920790a907b75 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:18.024777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.024777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.038835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097adad8-fe4b-4e42-8898-39c3cd9b3e4a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.044070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.044799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.045095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.045509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] PATH_INFO: `/v3/projects/e24320d28ddf494fbcc0aef7310378a1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.046741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24320d28ddf494fbcc0aef7310378a1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.048772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.097708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd067a-f5c1-4dcd-ac44-f60d1fa4f794 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.099034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:18.102554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.105724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.105724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.105724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.105724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.106574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b8b2c-d513-4b92-bc7f-1328ea7a6593 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.178847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24320d28ddf494fbcc0aef7310378a1', 'request_id': 'req-a995de34-b057-484e-92be-4d6ec80cce74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.185255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24320d28ddf494fbcc0aef7310378a1', 'request_id': 'req-a995de34-b057-484e-92be-4d6ec80cce74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.213654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1009/2002] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:18.214884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b3d3-7593-4f3f-b593-009951e4db3f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.215173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b3d3-7593-4f3f-b593-009951e4db3f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.215449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b3d3-7593-4f3f-b593-009951e4db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.234805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3475b3d3-7593-4f3f-b593-009951e4db3f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:18.241784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3475b3d3-7593-4f3f-b593-009951e4db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.266636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24320d28ddf494fbcc0aef7310378a1', 'request_id': 'req-a995de34-b057-484e-92be-4d6ec80cce74'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.278467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] The token cache is being invalidate because project e24320d28ddf494fbcc0aef7310378a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:18.278760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.284699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:18.284699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 994/2003] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:17 2022] DELETE /identity/v3/projects/e24320d28ddf494fbcc0aef7310378a1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:18.284699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.285788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.296732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1010/2004] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:18.298001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644ea2-fd3b-4ce5-a6ef-52ec807202b2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.298288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644ea2-fd3b-4ce5-a6ef-52ec807202b2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.298601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644ea2-fd3b-4ce5-a6ef-52ec807202b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.318082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-96644ea2-fd3b-4ce5-a6ef-52ec807202b2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:18.319481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-96644ea2-fd3b-4ce5-a6ef-52ec807202b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.328023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1011/2005] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:18.332292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393b7ef-ab38-4d07-a126-5e6406397179 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.332292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393b7ef-ab38-4d07-a126-5e6406397179 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.332292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393b7ef-ab38-4d07-a126-5e6406397179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.355017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0393b7ef-ab38-4d07-a126-5e6406397179 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:18.356375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0393b7ef-ab38-4d07-a126-5e6406397179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.361020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995de34-b057-484e-92be-4d6ec80cce74 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.364303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1012/2006] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:18.365666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.365857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.366073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.366211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.366732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.367019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.367296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.368005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.370631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.381061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 995/2007] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:18.381754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.389360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:18.390627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.392404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5885d-07f6-4440-850c-81c7b84ac399 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.399650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.400422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.400929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.401216 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.401914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:18.402001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1013/2008] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:18.402224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.402886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.403904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.414493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26230b5-fa2b-42dd-a3e5-012ca1233b88 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.422899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.432224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 996/2009] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:18.432805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.432805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.433119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.433214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.433214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.433214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.434560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6c5fc-6939-404f-8e47-14703a0a7876 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.442154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1014/2010] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:18.442301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.442437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.442549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.442549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.442549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.442549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:18.442549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.443487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.456929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:18.460697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 997/2011] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:18.460697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.466640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cc2ab-76d4-4c36-aae7-07eb368d6815 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.471083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.471217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c4f5-7d69-40e8-816a-247977d99a58 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.471655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.471945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.472336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.473028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.475036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.483076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.484048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.485102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.485838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] PATH_INFO: `/v3/projects/fcadcd45abe7437fae67f8ae305b78e5/users/5d9c833d928b4a9dac0ef2c8cf662043/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.504219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcadcd45abe7437fae67f8ae305b78e5, user_id=5d9c833d928b4a9dac0ef2c8cf662043, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.507493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.593512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 998/2012] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:18 2022] PUT /identity/v3/projects/fcadcd45abe7437fae67f8ae305b78e5/users/5d9c833d928b4a9dac0ef2c8cf662043/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:18.602841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.603690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1015/2013] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/projects => generated 359 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:18.604134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.668846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efbc9a-16dd-43b6-a835-d893cc7a61bb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.671291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34639-ebf4-4c29-aeae-ba6393743df5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.673744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.674390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.674675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.674958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.678171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.678320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.678320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.678320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.678320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.678687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a59ab0-c818-41e8-ab2a-93880f11147d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.678789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.680201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.784143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1016/2014] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/users => generated 280 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:18.785442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.785765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.786051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.814242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] MFA Rules not processed for user `5d9c833d928b4a9dac0ef2c8cf662043`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:18.815452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 999/2015] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/projects => generated 393 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:18.815588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:18.816451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.880960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cd3ad-fa1d-4275-9c31-d859752155d3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.886398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.887141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1017/2016] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:18.887282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.887483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.887866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.887970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.888674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.890502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.898503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb21e-1faf-45f7-941d-8b2e0f62c969 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.901922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1000/2017] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:18.902790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:18.904366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.905047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.905892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.906181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.906908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.910386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.913770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65825f-c9c5-42c9-bb2d-3247e362e4d3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:18.919141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:18.919784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:18.920072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:18.920354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:18.921062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:18.923196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:18.979230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:19.080307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1018/2018] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:19.080307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.088496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894183ee-edd4-4ce8-9916-55206cc94960 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.094135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.096175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.096175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.096175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.096175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.097710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.100371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1001/2019] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:18 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:19.101368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.111349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1019/2020] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:19.111349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.119957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc751-4fc4-4b6b-b008-db6a7f55def0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.174574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242ae78-ee78-4014-87a9-63d20681bf16 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.182231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.182231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.182231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.182231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.182634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.183100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.183759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.184388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.184704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.184989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.185669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.187667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.193266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1002/2021] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:19.194051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.204275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da640-4df1-452f-8d03-f87a8c27b221 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.209016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.209672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.209963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.210245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] PATH_INFO: `/v3/projects/03125129430645a6bc881464ff6f9ca3/users/f106bf71f4cf47509a547e1988b0e827/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.226252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03125129430645a6bc881464ff6f9ca3, user_id=f106bf71f4cf47509a547e1988b0e827, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.233906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.317457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:19.324471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1020/2022] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:19 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:19.325594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.334887 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22558-61dc-4921-abdf-5db0f523ab6a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.340433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.341051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.341375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.341662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.342484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.344124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.354922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1021/2023] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:19.355729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.365401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe000c-08b8-4812-bdcc-a6366f23b412 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.370479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.371084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.371368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.371659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.385901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.386044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1003/2024] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:19 2022] PUT /identity/v3/projects/03125129430645a6bc881464ff6f9ca3/users/f106bf71f4cf47509a547e1988b0e827/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:19.386044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.386192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe34e8d-d57d-4a25-9dd8-336aa6e6b2bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.386380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1022/2025] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:19.388226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.388473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.388757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.418259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] MFA Rules not processed for user `f106bf71f4cf47509a547e1988b0e827`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:19.419538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:19.437790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd667f2d-5ed9-4989-ae3f-79db8b666a1d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.442801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.443476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.443763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.444051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] PATH_INFO: `/v3/projects/bccc892b27d44d50979c5eaf5271b699/users/286d7480140646ba9dd1b36deddf1e21/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.461890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccc892b27d44d50979c5eaf5271b699, user_id=286d7480140646ba9dd1b36deddf1e21, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.466727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.488544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1023/2026] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:19.489572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.504034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5ef23-da7c-4d1c-b0a0-c04072b7feb2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.511159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.514265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.514265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.514265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] PATH_INFO: `/v3/projects/f1ab7956148d4ddba1358d3bbd01360d/users/afc538331a5342d1b1a6e0e5c45551c1/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.534763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1004/2027] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:19 2022] PUT /identity/v3/projects/bccc892b27d44d50979c5eaf5271b699/users/286d7480140646ba9dd1b36deddf1e21/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:19.535802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.538334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:19.548827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ab7956148d4ddba1358d3bbd01360d, user_id=afc538331a5342d1b1a6e0e5c45551c1, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.552229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.611836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be4664-11c3-4c5a-840f-0b0d8b91d0ec admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.616622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.617293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.617686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.617975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.688162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.689716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf32c34-e074-42d9-8968-3a2fb6ec581e service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.703857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1005/2028] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:19.707430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1024/2029] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:19 2022] PUT /identity/v3/projects/f1ab7956148d4ddba1358d3bbd01360d/users/afc538331a5342d1b1a6e0e5c45551c1/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:19.707733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.708435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.708435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.709687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.767784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] MFA Rules not processed for user `286d7480140646ba9dd1b36deddf1e21`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:19.768121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:19.832077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8463f9-dff7-42da-b4a3-28d197630df0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.848366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.849069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.849365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.849652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.850746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.852383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.857879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:19.878439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1025/2030] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:19.878439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.880441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1006/2031] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:19.881436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.890272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e636e33-0211-4583-8ed9-c51df33029af admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.894824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147159ec-14c8-40b9-bacb-990f3e041808 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.898860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.899504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.900367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.900583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.900694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] PATH_INFO: `/v3/users/f106bf71f4cf47509a547e1988b0e827` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.901267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.901557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.901893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f106bf71f4cf47509a547e1988b0e827)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.902662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] PATH_INFO: `/v3/projects/f1ab7956148d4ddba1358d3bbd01360d/users/afc538331a5342d1b1a6e0e5c45551c1/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.905123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.905279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ab7956148d4ddba1358d3bbd01360d, user_id=afc538331a5342d1b1a6e0e5c45551c1, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:19.907846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:19.918918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1007/2032] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:19 2022] PUT /identity/v3/projects/f1ab7956148d4ddba1358d3bbd01360d/users/afc538331a5342d1b1a6e0e5c45551c1/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:19.922499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:19.991937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb7c5-bb3b-4959-a510-3148308ab6e0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:19.997191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:19.997823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:19.998114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:19.998555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:19.999234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.001165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5fd2d-7277-4e04-8831-00eb73a11982 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.117728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1008/2033] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:19 2022] POST /identity/v3/projects => generated 393 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:20.117728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.117728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.117728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.148661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] MFA Rules not processed for user `afc538331a5342d1b1a6e0e5c45551c1`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:20.150654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.153481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f106bf71f4cf47509a547e1988b0e827'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:20.154272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f106bf71f4cf47509a547e1988b0e827'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:20.219726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1009/2034] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:20.220629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.226181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:20.260923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f106bf71f4cf47509a547e1988b0e827'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:20.274054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f106bf71f4cf47509a547e1988b0e827'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:20.275524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f106bf71f4cf47509a547e1988b0e827'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:20.285764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca242f-a26e-4ce6-a492-3bf08b4e7517 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.289073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1026/2035] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:19 2022] DELETE /identity/v3/users/f106bf71f4cf47509a547e1988b0e827 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:20.290281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.291792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.292907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.293201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.293473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.293866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.294155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.296265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05774-409c-4bbf-af86-e89cb807d79c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.353701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f7f6-945b-4b5c-a612-6c37f0db7361 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.357942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.358587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.358947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.359236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.365390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.422297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1010/2036] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:20 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:20.424200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795936-8762-4b18-abfc-687ded317eff None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.424494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795936-8762-4b18-abfc-687ded317eff None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.424775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795936-8762-4b18-abfc-687ded317eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.435588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.436948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.446663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d6795936-8762-4b18-abfc-687ded317eff None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:20.447971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d6795936-8762-4b18-abfc-687ded317eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.450498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1027/2037] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:20.451841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.454100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.523809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1011/2038] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:20.528468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60751-14c2-4709-a2a6-0e795417f9f8 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.533628 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.534269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.534573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.534680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795936-8762-4b18-abfc-687ded317eff None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.534856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.535565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.536238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d6795936-8762-4b18-abfc-687ded317eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.537513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.549637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1028/2039] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:20.558666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.569335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f43e0-9e02-4858-a4b4-f51de976309c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.575584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.576241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.576584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.576928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] PATH_INFO: `/v3/projects/56791cf59d5b459b90d7abadc059c246/users/1c7243d3f21141b1aea34b965cb0a697/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.592856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=56791cf59d5b459b90d7abadc059c246, user_id=1c7243d3f21141b1aea34b965cb0a697, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.595834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.600716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795936-8762-4b18-abfc-687ded317eff None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.604505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.605130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.605420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.605701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.615025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.616575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.626883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1012/2040] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:20.633674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.643921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d00ca-004f-470d-8562-954e4b2131e6 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.646404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:20.649688 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.651149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.658995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.658995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.658995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:20.716535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.718149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.727765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1013/2041] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:20.735413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1029/2042] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:20 2022] PUT /identity/v3/projects/56791cf59d5b459b90d7abadc059c246/users/1c7243d3f21141b1aea34b965cb0a697/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:20.737489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.753618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887fca-e8ae-4c73-b879-a8eb31e3d6ec service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.808384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.809038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.809326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.809607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.810315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.812681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4af56-2b24-4220-b5a7-219583d1304a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.827446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1014/2043] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:20.830433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:20.844537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c416577-7a5b-4d6f-b60b-c1e9941405ae admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:20.850214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:20.850912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.851199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.851495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] PATH_INFO: `/v3/projects/56791cf59d5b459b90d7abadc059c246/users/1c7243d3f21141b1aea34b965cb0a697/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.853908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56791cf59d5b459b90d7abadc059c246, user_id=1c7243d3f21141b1aea34b965cb0a697, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:20.856666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:20.870404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1030/2044] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:20 2022] PUT /identity/v3/projects/56791cf59d5b459b90d7abadc059c246/users/1c7243d3f21141b1aea34b965cb0a697/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:20.879819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:20.879819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:20.880063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:20.921438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] MFA Rules not processed for user `1c7243d3f21141b1aea34b965cb0a697`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:20.924834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.029508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1015/2045] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:21.104117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.106513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.196208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba9907-00b7-4481-855a-f04c11721001 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.200499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.201112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.201391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.201666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.210179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:21.213533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.215257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.225457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1031/2046] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:21.247132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.249171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.365765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5cc9e-8d97-4fb4-8ab2-4af57511153d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.371463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.372080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.372365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.372646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] PATH_INFO: `/v3/projects/03125129430645a6bc881464ff6f9ca3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.373848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03125129430645a6bc881464ff6f9ca3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.375881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.418836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.427633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99f4-7cd9-4424-8b0e-4c0d364f3147 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.432758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.432758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.433197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.433658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.435890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.503419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:21.506598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03125129430645a6bc881464ff6f9ca3', 'request_id': 'req-249a59fd-bcc5-46b9-86a9-21d633ad5d70'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:21.512160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.513252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03125129430645a6bc881464ff6f9ca3', 'request_id': 'req-249a59fd-bcc5-46b9-86a9-21d633ad5d70'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:21.513625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1ba27-98d5-4091-9bfc-04b20cee2a53 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.523233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1032/2047] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:21.586826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03125129430645a6bc881464ff6f9ca3', 'request_id': 'req-249a59fd-bcc5-46b9-86a9-21d633ad5d70'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:21.600099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] The token cache is being invalidate because project 03125129430645a6bc881464ff6f9ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:21.600400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:21.601058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-249a59fd-bcc5-46b9-86a9-21d633ad5d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:21.602623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1016/2048] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:21 2022] DELETE /identity/v3/projects/03125129430645a6bc881464ff6f9ca3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:21.608703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.611292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.611292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.633331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:21.634695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.671021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:21.705113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1033/2049] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:21.713515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.713801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.714108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.735604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:21.736939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.745434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1017/2050] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:21.749809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.760081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45a01-4c1f-4c9d-b58d-93069c7295c1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.765202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.765833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.766117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.766427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.767129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:21.767439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.769720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.783900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1034/2051] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:21.788569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.803593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3689-dbbf-48cf-a8f0-e58eeb00be89 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.822149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.823081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.828094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.828094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.828094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.828094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.884572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:21.886432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:21.899335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987b2d-d5fe-4763-b5e5-07b5a97d444c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:21.905062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:21.905706 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:21.906083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:21.906382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] PATH_INFO: `/v3/users/5d9c833d928b4a9dac0ef2c8cf662043` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:21.907611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9c833d928b4a9dac0ef2c8cf662043)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:21.914193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:21.949980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1018/2052] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:21 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:21.952460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.037145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8c60-887a-4eff-85e3-0087f40e84d6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.042747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.043378 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.043663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.043986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.044661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.046650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.158858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9c833d928b4a9dac0ef2c8cf662043'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:22.159579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9c833d928b4a9dac0ef2c8cf662043'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:22.184212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1019/2053] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:21 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:22.185414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.196145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8735fdb-4692-4ae8-a9cb-a99d7d288b9a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.201773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.202432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.202729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.206791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.206791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.206791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.217301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1020/2054] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:22.222215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.232589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd3e9-43f7-44a4-9640-e21b81f2f726 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.237465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.238094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.238416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.238700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.239515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.241112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.257165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1021/2055] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:22.261319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.265035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9c833d928b4a9dac0ef2c8cf662043'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:22.293088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ebcb0-723f-4631-8b3a-15ddd9b5b457 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.315418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9c833d928b4a9dac0ef2c8cf662043'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:22.315724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.316423 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.316707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.316901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c8e773bd-1cb6-4439-838a-19e99ce46a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9c833d928b4a9dac0ef2c8cf662043'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:22.319146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] PATH_INFO: `/v3/projects/bd26d202c7b7438897ffd65805b64fd3/users/c8af8f9a31c845f29f225141220fffbc/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.343729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1035/2056] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:21 2022] DELETE /identity/v3/users/5d9c833d928b4a9dac0ef2c8cf662043 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:22.345371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd26d202c7b7438897ffd65805b64fd3, user_id=c8af8f9a31c845f29f225141220fffbc, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.347048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:22.347210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.347791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.348508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.353623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06a030-5fe4-4db6-ae91-183b5eb3d8d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.405565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:22.405565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:22.441736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1022/2057] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:22 2022] PUT /identity/v3/projects/bd26d202c7b7438897ffd65805b64fd3/users/c8af8f9a31c845f29f225141220fffbc/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:22.455788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.456093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.456556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.495781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] MFA Rules not processed for user `c8af8f9a31c845f29f225141220fffbc`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:22.497241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:22.499928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1036/2058] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:22.511010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.514670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:22.573163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1023/2059] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:22.579434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428d433-1343-4127-827a-709aa9876ea4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.579604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.581020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:22.654120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.654917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.655266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.655606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.667330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.668922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.673870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c276f2-0024-4846-b786-9cf05f3fbd53 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.678909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.679450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1037/2060] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:22.679567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.679909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.680230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.681525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.683541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.704272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:22.788634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1024/2061] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:22 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:22.793591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.803432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04a190-139f-4b97-9c1c-f1fa52adcbea service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:22.864845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:22.865559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:22.865853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:22.866140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:22.867042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:22.869875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:22.978254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1038/2062] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:22 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:22.983058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:22.999800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87ce9-58ee-4011-966c-ee860fe400c0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.005307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.006086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.006414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.006764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.007552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.010705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.024593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1025/2063] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:23.028711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.044335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267c3c6-2b9c-42fe-aaaf-ea73fd1480e3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.051131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.051964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.052653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.052653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.053412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.055204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.068523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1039/2064] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:23.073816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.086673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03be46-a8d9-4536-b1f8-65ad9580ec2f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.092434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.093212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.093594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.093946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] PATH_INFO: `/v3/projects/77d68b6b47ea4ba6827e964c67307e76/users/2e62de83fcf547f2a5f01fafce26e5e0/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.112206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d68b6b47ea4ba6827e964c67307e76, user_id=2e62de83fcf547f2a5f01fafce26e5e0, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.115164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c239f9-fadb-4b00-b798-e56126a99da1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.181636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.185664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:23.203324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf6751-1dcc-4c1f-82ee-a329b08ddf22 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.206322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1026/2065] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:23 2022] PUT /identity/v3/projects/77d68b6b47ea4ba6827e964c67307e76/users/2e62de83fcf547f2a5f01fafce26e5e0/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:23.220670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.221028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.221440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.250454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] MFA Rules not processed for user `2e62de83fcf547f2a5f01fafce26e5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:23.251985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:23.273347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.273994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.274280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.275112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] PATH_INFO: `/v3/projects/fcadcd45abe7437fae67f8ae305b78e5` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.276522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcadcd45abe7437fae67f8ae305b78e5)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.278477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.324296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1027/2066] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:23.328811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.340176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dabe9-675b-44b4-b6de-eef918f0f9a8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.344913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.350067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.350067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.350067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.350067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.350067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.417634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcadcd45abe7437fae67f8ae305b78e5', 'request_id': 'req-68268232-d8d8-4472-8210-b435d1ad8a32'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.424087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcadcd45abe7437fae67f8ae305b78e5', 'request_id': 'req-68268232-d8d8-4472-8210-b435d1ad8a32'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.439052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcadcd45abe7437fae67f8ae305b78e5', 'request_id': 'req-68268232-d8d8-4472-8210-b435d1ad8a32'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.449317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1028/2067] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:23 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:23.453200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.487662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:23.523692 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] The token cache is being invalidate because project fcadcd45abe7437fae67f8ae305b78e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:23.524270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.525388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.527186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1040/2068] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:23 2022] DELETE /identity/v3/projects/fcadcd45abe7437fae67f8ae305b78e5 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:23.530951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.532417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:23.541558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c338a-8a83-4317-ae5d-53bb51157a06 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.546436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.546985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.547274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.547663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.548229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.550159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.605845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268232-d8d8-4472-8210-b435d1ad8a32 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.610891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.611532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.611823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.612101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] PATH_INFO: `/v3/users/afc538331a5342d1b1a6e0e5c45551c1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.613278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc538331a5342d1b1a6e0e5c45551c1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.615297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.723620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1029/2069] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:23 2022] POST /identity/v3/users => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:23.726565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.729084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:23.745000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9181c5-0c59-489d-8e7c-c077554fb36a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.751122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.751915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.752267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.752640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.753435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.755418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.782682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1030/2070] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:23.783877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:23.796018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630adf-83bb-450f-9a31-ab7ce7d465bd admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:23.801374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:23.802031 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:23.802219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:23.802517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] PATH_INFO: `/v3/projects/aa16f0552e824785b660fa0ef991a0e0/users/ae82ad54ccc744b0beb4d9aebd97e336/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:23.820122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa16f0552e824785b660fa0ef991a0e0, user_id=ae82ad54ccc744b0beb4d9aebd97e336, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:23.823211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:23.897542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc538331a5342d1b1a6e0e5c45551c1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.898210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc538331a5342d1b1a6e0e5c45551c1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:23.993533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc538331a5342d1b1a6e0e5c45551c1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:24.002611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1031/2071] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:23 2022] PUT /identity/v3/projects/aa16f0552e824785b660fa0ef991a0e0/users/ae82ad54ccc744b0beb4d9aebd97e336/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:24.007272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:24.012508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc538331a5342d1b1a6e0e5c45551c1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:24.013223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc538331a5342d1b1a6e0e5c45551c1'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:24.026276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1041/2072] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:23 2022] DELETE /identity/v3/users/afc538331a5342d1b1a6e0e5c45551c1 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:24.077813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbc2a-ac25-4157-a6ac-c3d25d880a40 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:24.141211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:24.141847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:24.142135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:24.142496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:24.143399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:24.145140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad20b75-b2a4-4d02-806d-a941169da7c3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:24.155323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1032/2073] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:24.159773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:24.161243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:24.174815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef2fa3-9efe-46e3-ae00-030723f2cbc0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:24.179414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:24.180052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:24.180341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:24.180619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] PATH_INFO: `/v3/projects/aa16f0552e824785b660fa0ef991a0e0/users/ae82ad54ccc744b0beb4d9aebd97e336/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:24.183105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa16f0552e824785b660fa0ef991a0e0, user_id=ae82ad54ccc744b0beb4d9aebd97e336, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:24.185767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:24.197133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1042/2074] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:24 2022] PUT /identity/v3/projects/aa16f0552e824785b660fa0ef991a0e0/users/ae82ad54ccc744b0beb4d9aebd97e336/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:24.207844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:24.208513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:24.208513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:24.240296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] MFA Rules not processed for user `ae82ad54ccc744b0beb4d9aebd97e336`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:24.241544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:24.329142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1033/2075] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:24.340116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:24.340116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:24.421575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c9373-6ae5-48f6-8480-4848d6a96ff5 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:24.426271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:24.427214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:24.427443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:24.427732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:24.429731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:24.499361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:24.500812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70011a-5179-469c-8d28-b87c1bef9c07 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:24.510549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1043/2076] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:24.617547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:24.623282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:24.641965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:24.722255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea94bb7-b855-4183-94ea-a47b9b4450ba None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:24.728663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:24.729386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:24.729699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:24.729986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] PATH_INFO: `/v3/projects/f1ab7956148d4ddba1358d3bbd01360d` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:24.731277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ab7956148d4ddba1358d3bbd01360d)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:24.733348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:24.748831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:24.916590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ab7956148d4ddba1358d3bbd01360d', 'request_id': 'req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:24.922301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ab7956148d4ddba1358d3bbd01360d', 'request_id': 'req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:25.030530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ab7956148d4ddba1358d3bbd01360d', 'request_id': 'req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:25.043451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] The token cache is being invalidate because project f1ab7956148d4ddba1358d3bbd01360d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:25.043945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:25.044463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9ac09ddd-5a94-4a68-9b35-aaa31c740df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:25.046200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1034/2077] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:24 2022] DELETE /identity/v3/projects/f1ab7956148d4ddba1358d3bbd01360d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:25.051118 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.051395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.051680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.072487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:25.073851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:25.146993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1044/2078] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:25.155373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.155877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.155970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.176421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:25.177671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:25.186079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1035/2079] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:25.190501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.201273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0476e8-dc00-4587-9b25-467d6572e1fd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.206107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.206767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.207056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.207336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.208040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:25.208351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.209959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.221832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1045/2080] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:25.226231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.236598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3cc28-40a8-4046-abf9-a89503f41031 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.241399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.241998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.242278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.242736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.244447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.246519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.428538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1036/2081] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:25.428929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.448820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba77edc-09c4-4240-b0dc-3dd019cb607f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.529595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.530406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.530863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.531355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.532095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.534425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.536656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:25.646968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.654860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1046/2082] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:25 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:25.660661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ed5f-3759-44f6-a973-20eb08786a22 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.665917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.666763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.667109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.675146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.675146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.675146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.685222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1037/2083] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:25.686161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.709986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2731a-f588-47fc-adcd-3192dfca6860 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.709986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.709986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.709986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.710327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.711517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.714003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2871d25-acce-476f-a4e3-fa4b1866f905 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.726965 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1047/2084] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:25.732267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:25.744024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83462b87-4d76-4c24-8cec-71824925798a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:25.750708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:25.751398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.751690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.751979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] PATH_INFO: `/v3/projects/59b0a55a29a24b0ca15868fb25a73ca4/users/8835b71fe1324373bfd00e30114c5d0b/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.771796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b0a55a29a24b0ca15868fb25a73ca4, user_id=8835b71fe1324373bfd00e30114c5d0b, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:25.775366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:25.888503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1038/2085] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:25 2022] PUT /identity/v3/projects/59b0a55a29a24b0ca15868fb25a73ca4/users/8835b71fe1324373bfd00e30114c5d0b/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:25.890257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:25.890591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:25.890873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:25.921252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] MFA Rules not processed for user `8835b71fe1324373bfd00e30114c5d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:25.922816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:26.007961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1048/2086] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:26.013678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.015813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:26.079993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545e4cb-461e-4980-986f-7f57459b4e38 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.084847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.085471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.085758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.086037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.086752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.089232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.200168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1039/2087] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:26 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:26.204797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.219289 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde8e0-32cd-4ef6-a35c-e2d9d56642ee None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.285865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.287214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.293332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.293332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.293332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.293332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.378319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1049/2088] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:26 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:26.382563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.393822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcb990-8032-492f-8bbb-62ff597800c0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.398993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.400623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.401180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.401622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.402380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.406057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.419966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1040/2089] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:26.421229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.433095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc640-2d4e-4025-aa4e-ed86f5b4f1ed admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.437789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.438790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.440148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.441032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] PATH_INFO: `/v3/projects/63715cfd5dd64054a90a060d74f580bb/users/4aa59ff023ed4386a469d503d0fb239e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.457930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=63715cfd5dd64054a90a060d74f580bb, user_id=4aa59ff023ed4386a469d503d0fb239e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.461035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.576175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1050/2090] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:26 2022] PUT /identity/v3/projects/63715cfd5dd64054a90a060d74f580bb/users/4aa59ff023ed4386a469d503d0fb239e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:26.580270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.590936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca228201-6bae-4413-9fc2-8ce324f69d04 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.651025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.651648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.651933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.652220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.652913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.654576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef75c9b-ff81-4ef2-8b83-fcc1711b8149 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.665975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1041/2091] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:26.670480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:26.681718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac769e3-4983-4bbc-829a-e967d1fce5ad admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:26.687414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:26.688104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.688471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.688759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] PATH_INFO: `/v3/projects/63715cfd5dd64054a90a060d74f580bb/users/4aa59ff023ed4386a469d503d0fb239e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.690837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63715cfd5dd64054a90a060d74f580bb, user_id=4aa59ff023ed4386a469d503d0fb239e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:26.693520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:26.704564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1051/2092] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:26 2022] PUT /identity/v3/projects/63715cfd5dd64054a90a060d74f580bb/users/4aa59ff023ed4386a469d503d0fb239e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:26.717665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:26.717907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:26.720102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:26.746765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] MFA Rules not processed for user `4aa59ff023ed4386a469d503d0fb239e`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:26.748045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:26.828319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1042/2093] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:27.204507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:27.206310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:27.265226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:27.267042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:27.275694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad61c6-48ba-46de-afe0-fd4e6a0e7bc4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:27.280617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:27.281255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:27.281537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:27.281812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] PATH_INFO: `/v3/users/286d7480140646ba9dd1b36deddf1e21` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:27.283183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=286d7480140646ba9dd1b36deddf1e21)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:27.285230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:27.326983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b004cc-4964-441f-83fc-9bf449c091e8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:27.330651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:27.331266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:27.331551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:27.331829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:27.339266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:27.390954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:27.392415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4c316-c697-4afd-9c5c-30045dce549e service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:27.402010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1043/2094] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:27.533898 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286d7480140646ba9dd1b36deddf1e21'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:27.533999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286d7480140646ba9dd1b36deddf1e21'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:27.609347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286d7480140646ba9dd1b36deddf1e21'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:27.622287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286d7480140646ba9dd1b36deddf1e21'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:27.623070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286d7480140646ba9dd1b36deddf1e21'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:27.635865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1052/2095] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:27 2022] DELETE /identity/v3/users/286d7480140646ba9dd1b36deddf1e21 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:27.640548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:27.640848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:27.640995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:27.641141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:27.662156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:27.663729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:27.723913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1044/2096] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:27.733579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:27.735067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:27.792435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09334982-8ea7-45d5-8594-25cc15bd681b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:27.795982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:27.796602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:27.796889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:27.797170 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:27.806689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:27.808183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:27.817920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1053/2097] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:27.944528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:28.167217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:28.304956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:28.307148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:28.350537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:28.352189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:28.367196 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553c75-b75b-455a-a7ae-548db2ed2fb7 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:28.383849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e65e87-3378-4cd7-aace-49282f96d735 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:28.394220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:28.395098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:28.395554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:28.395988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:28.398247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:28.401751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:28.401874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:28.401874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:28.403053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] PATH_INFO: `/v3/projects/bccc892b27d44d50979c5eaf5271b699` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:28.404536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccc892b27d44d50979c5eaf5271b699)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:28.406882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:28.424162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:28.466540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:28.467455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:28.477358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1045/2098] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:28.545012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccc892b27d44d50979c5eaf5271b699', 'request_id': 'req-12cc72be-dfdf-4455-8ffe-a4082e44e966'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:28.557590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccc892b27d44d50979c5eaf5271b699', 'request_id': 'req-12cc72be-dfdf-4455-8ffe-a4082e44e966'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:28.572047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccc892b27d44d50979c5eaf5271b699', 'request_id': 'req-12cc72be-dfdf-4455-8ffe-a4082e44e966'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:28.686000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] The token cache is being invalidate because project bccc892b27d44d50979c5eaf5271b699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:28.686213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:28.686822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:28.688335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1054/2099] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:28 2022] DELETE /identity/v3/projects/bccc892b27d44d50979c5eaf5271b699 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:28.692690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:28.704023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fbdfe-92d7-48de-b04d-000f9dc958f2 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:28.763553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:28.764200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:28.764600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:28.764890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] PATH_INFO: `/v3/users/1c7243d3f21141b1aea34b965cb0a697` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:28.766095 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7243d3f21141b1aea34b965cb0a697)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:28.768119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:29.123372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7243d3f21141b1aea34b965cb0a697'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:29.124060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7243d3f21141b1aea34b965cb0a697'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:29.124680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7243d3f21141b1aea34b965cb0a697'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:29.137303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7243d3f21141b1aea34b965cb0a697'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:29.221111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7243d3f21141b1aea34b965cb0a697'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:29.234461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1046/2100] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:28 2022] DELETE /identity/v3/users/1c7243d3f21141b1aea34b965cb0a697 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:29.252639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:29.366465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:29.581046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:29.785078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:29.787327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:29.800217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc72be-dfdf-4455-8ffe-a4082e44e966 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:29.885870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:29.886600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:29.886950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:29.887214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] PATH_INFO: `/v3/projects/56791cf59d5b459b90d7abadc059c246` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:29.888624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56791cf59d5b459b90d7abadc059c246)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:29.890840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.001926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.003445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.017018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c659b-63d1-414a-89c0-67e775995973 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.021795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.022474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.022755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.023043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] PATH_INFO: `/v3/users/019e016faa3b4953b4f1e253253fcb28` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.024250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019e016faa3b4953b4f1e253253fcb28)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.026417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.101650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56791cf59d5b459b90d7abadc059c246', 'request_id': 'req-8d77487b-13e2-452f-a2be-753fa8483951'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.107428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:30.127165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56791cf59d5b459b90d7abadc059c246', 'request_id': 'req-8d77487b-13e2-452f-a2be-753fa8483951'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.135959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56791cf59d5b459b90d7abadc059c246', 'request_id': 'req-8d77487b-13e2-452f-a2be-753fa8483951'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.199318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:30.240338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] The token cache is being invalidate because project 56791cf59d5b459b90d7abadc059c246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:30.240627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.241307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8d77487b-13e2-452f-a2be-753fa8483951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.243944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1055/2101] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:29 2022] DELETE /identity/v3/projects/56791cf59d5b459b90d7abadc059c246 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:30.248708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429c80-f2cf-42cd-b02c-e2fa8a73988a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.249112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429c80-f2cf-42cd-b02c-e2fa8a73988a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.249515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb429c80-f2cf-42cd-b02c-e2fa8a73988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.250807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019e016faa3b4953b4f1e253253fcb28'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.251508 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019e016faa3b4953b4f1e253253fcb28'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.252242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019e016faa3b4953b4f1e253253fcb28'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.266659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019e016faa3b4953b4f1e253253fcb28'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.277701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fb429c80-f2cf-42cd-b02c-e2fa8a73988a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:30.279001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fb429c80-f2cf-42cd-b02c-e2fa8a73988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.344742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1056/2102] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:30.353783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.354079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.354438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.359821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e9030d7f-a71e-45cf-99ff-16f8786369c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019e016faa3b4953b4f1e253253fcb28'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:30.378520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1047/2103] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:30 2022] DELETE /identity/v3/users/019e016faa3b4953b4f1e253253fcb28 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:30.379108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:30.379108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.380554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.381041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.381395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.404204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:30.405645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.443705 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1057/2104] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:30.448195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.457210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03969f0-ffa5-4051-903b-899c2f630a36 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.461982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.462886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.463175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.463456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.464154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:30.464466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.466072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.470179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1048/2105] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:30.480640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.483995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1058/2106] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:30.484100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.484850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.496183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5245be6-aea8-495b-9747-22838a8cc6a3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.501089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.501782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.502117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.502488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.503220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.505253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.548518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b4c48-0e48-4ea5-9415-ca84ec339a79 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.553377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.553944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.554230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.554541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.564784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.566612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.576512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1049/2107] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:30.640719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1059/2108] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:30 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:30.645316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.667536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d8323-3808-48d4-8d4c-25720c554bce service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.739400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.740098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.740446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.740802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.741473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.744244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.775293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.777493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.876714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1050/2109] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:30 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:30.885597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.897501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c2e9f-c849-4311-a16f-2cf9fd9f3f99 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.922618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.923418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.924646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.926399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.928810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e09a9-747b-4b74-a771-41e7272b7cde admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:30.942296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:30.942532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:30.950155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:30.951529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:30.954244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:30.954244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:30.954244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:30.956663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:30.976063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1051/2110] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:30.980278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:30.995678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e6e-b5c9-4f25-a452-0aa228d0a0bf admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.004664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.005778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.006307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.006737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.008090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.010388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.028795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1052/2111] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:31.036500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.036930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.038666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.049519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1060/2112] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:31.050074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceb26c-ffd5-4b87-9406-37f001b7c827 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.059850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.059850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.059850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.059850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] PATH_INFO: `/v3/projects/deb29d922b58443cb7d2fc72a965ef82/users/6aa2dd17e6c14d079800f451cf3c8392/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.075308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb29d922b58443cb7d2fc72a965ef82, user_id=6aa2dd17e6c14d079800f451cf3c8392, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.081082 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ede55-fae2-4273-98ed-89baa4de4c1e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.199476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.210536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1053/2113] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:31 2022] PUT /identity/v3/projects/deb29d922b58443cb7d2fc72a965ef82/users/6aa2dd17e6c14d079800f451cf3c8392/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:31.214364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:31.221361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.221662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.222463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.231723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744880e5-1e03-410e-bae3-40b1ab908650 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.266744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] MFA Rules not processed for user `6aa2dd17e6c14d079800f451cf3c8392`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:31.268048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:31.283902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:31.304518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.305574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.306062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.307190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] PATH_INFO: `/v3/projects/015548578c01407490700514ec75c86f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.307828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015548578c01407490700514ec75c86f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.309852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.354820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1054/2114] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:31.359659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.377435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc22e0-9e71-4991-a784-69cb0601e4f0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.391092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.391092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.391092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.391092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.392081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.392415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.473930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015548578c01407490700514ec75c86f', 'request_id': 'req-1165966a-93ae-4e82-9fd3-85f7ccf8f370'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:31.480776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015548578c01407490700514ec75c86f', 'request_id': 'req-1165966a-93ae-4e82-9fd3-85f7ccf8f370'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:31.487144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015548578c01407490700514ec75c86f', 'request_id': 'req-1165966a-93ae-4e82-9fd3-85f7ccf8f370'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:31.521876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1055/2115] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:31.526147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.578524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:31.591763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec15fd-e716-4cf8-a5f2-47fa69e4c907 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.596570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.597182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.597471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.597753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.598448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.600444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.638336 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] The token cache is being invalidate because project 015548578c01407490700514ec75c86f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:31.638723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:31.639330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-1165966a-93ae-4e82-9fd3-85f7ccf8f370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:31.640848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1061/2116] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:31 2022] DELETE /identity/v3/projects/015548578c01407490700514ec75c86f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:31.645460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc51b5-696d-4fd8-9f55-88a9dee7f918 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.645752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc51b5-696d-4fd8-9f55-88a9dee7f918 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.646028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc51b5-696d-4fd8-9f55-88a9dee7f918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.665051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a3dc51b5-696d-4fd8-9f55-88a9dee7f918 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:31.666431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a3dc51b5-696d-4fd8-9f55-88a9dee7f918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:31.674310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1062/2117] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:31.683576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.683867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.684147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.704208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:31.705401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:31.714954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1063/2118] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:31.717209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1056/2119] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:31.720661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.721463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.723181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:31.732967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b3179-db41-4b2d-abd4-e51d53c48f88 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.735902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a438fb0-d625-4a2d-b41a-5a081d29a6c9 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.737947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.738908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.739201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.739574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.740182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:31.740477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.740887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.741512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.741802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.742083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.742175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.749208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.749208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.755804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1057/2120] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:31.763015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1064/2121] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:31.763288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.774272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f4ea4-2108-446a-96f4-5ecb12899308 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.775841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.779204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.779860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.780169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.780419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] PATH_INFO: `/v3/projects/518ea4f80df4498d8d6e059f113defa4/users/3bca1bf9118047769fc0ebef22a8e6aa/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.785653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d004df-caa8-4f93-91ec-9c336144a70f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:31.791926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:31.792549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:31.792853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:31.797178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:31.797178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.797178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.797445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=518ea4f80df4498d8d6e059f113defa4, user_id=3bca1bf9118047769fc0ebef22a8e6aa, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:31.799227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:31.910151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1058/2122] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:31 2022] PUT /identity/v3/projects/518ea4f80df4498d8d6e059f113defa4/users/3bca1bf9118047769fc0ebef22a8e6aa/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:31.914059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.915221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1065/2123] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:31.918804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:31.986416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10deb-7ddc-4381-8a22-7ada871f9dae admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.006747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0979e-f4af-41bf-a236-8f74fe4e4705 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.012917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.013555 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.013843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.014458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.015284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.017913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.062700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.062700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.062700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.062700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.063417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.065142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.076684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1059/2124] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:32.080786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.094976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3da05-15db-48f3-8003-7fbc6849919a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.100634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.101879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.102379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.102697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] PATH_INFO: `/v3/projects/518ea4f80df4498d8d6e059f113defa4/users/3bca1bf9118047769fc0ebef22a8e6aa/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.104666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518ea4f80df4498d8d6e059f113defa4, user_id=3bca1bf9118047769fc0ebef22a8e6aa, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.110540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9cb14-6b03-465a-bdea-ba8123591156 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.126225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1060/2125] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:32 2022] PUT /identity/v3/projects/518ea4f80df4498d8d6e059f113defa4/users/3bca1bf9118047769fc0ebef22a8e6aa/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:32.129806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.130217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.130982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.140340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1066/2126] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:31 2022] POST /identity/v3/users => generated 284 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:32.144619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.158982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64af2d3-372d-4f29-95be-d21331e3292b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.173273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] MFA Rules not processed for user `3bca1bf9118047769fc0ebef22a8e6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:32.175235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:32.249143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.250176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.250957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.251766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.252629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.256242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.273379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1061/2127] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:32.273976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1067/2128] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:32.278401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.289432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2b27-9f77-4855-b0b6-748faaefc41d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.294629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.295280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.295568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.298833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.298833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.298833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c4b97-b271-47fc-af7d-f5eec4ff000f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.310402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1062/2129] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:32.313684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.324627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac5023-f485-4d52-944b-251ec999e79e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.329749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.330551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.330840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.331122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] PATH_INFO: `/v3/projects/4a0086c45596414f95f854e8bc79bba0/users/1735d20129aa4b11bc8afc42340f6f5c/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.346407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0086c45596414f95f854e8bc79bba0, user_id=1735d20129aa4b11bc8afc42340f6f5c, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.349249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.438780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1068/2130] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:32 2022] PUT /identity/v3/projects/4a0086c45596414f95f854e8bc79bba0/users/1735d20129aa4b11bc8afc42340f6f5c/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:32.449777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.450300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.450417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.478407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] MFA Rules not processed for user `1735d20129aa4b11bc8afc42340f6f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:32.479666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:32.529374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.531178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:32.549832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1063/2131] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:32.562686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:32.564090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:32.591393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51e5b-b655-478d-ac00-2fa09785de35 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.596008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.596615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.596931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.597208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.652281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d8eb-42db-4a45-b751-2a02032c7a16 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:32.655933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:32.656565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:32.656844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:32.657128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:32.666114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.667766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.675120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:32.676394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1064/2132] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:32.678369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:32.688768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1069/2133] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:32.761273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:33.331767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:33.880839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:33.887931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8d033-099d-4ad3-b2fd-51a81a3b7835 service glance] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:33.893020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:33.894648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:33.894975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:33.895866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:33.898465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:33.977931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:33.977931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:33.990602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1065/2134] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:34.166957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:34.900702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:34.905606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:34.918370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474b9ab-7e09-43f8-8f3b-3e18a9b4e6be service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:34.927283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:34.928020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:34.928447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:34.929086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:34.933524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:35.022880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:35.024234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974740e-eea6-47d9-a63f-0ce85ba89157 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:35.036189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1070/2135] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:35.041250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:35.755520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:35.947422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:36.342560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:36.344381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:36.418562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10c66-5740-4240-b511-c701ee8ee4cb service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:36.423436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:36.424179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:36.424469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:36.424797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] PATH_INFO: `/v3/users/0c5e2ea8311b4eb78963fddd85f15270` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:36.425986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5e2ea8311b4eb78963fddd85f15270)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:36.427984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:36.658081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5e2ea8311b4eb78963fddd85f15270'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:36.658850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5e2ea8311b4eb78963fddd85f15270'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:36.659487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5e2ea8311b4eb78963fddd85f15270'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:36.672357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5e2ea8311b4eb78963fddd85f15270'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:36.735117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5e2ea8311b4eb78963fddd85f15270'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:36.749699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1066/2136] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:36 2022] DELETE /identity/v3/users/0c5e2ea8311b4eb78963fddd85f15270 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:36.754161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:36.754985 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:36.755540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:36.791399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:36.791399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:36.844078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:36.938544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1071/2137] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:36.967836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:36.969617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:37.104111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3d90d-d581-4e22-b0da-4bf698c1b3a1 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:37.104111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:37.104111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:37.104111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:37.104452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:37.104452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:37.104452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:37.104452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1067/2138] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:37.132065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:37.247776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:37.249366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:37.262317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dff8e2-cced-4e3c-b8dc-d99d7a50e879 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:37.267223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:37.267854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:37.268142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:37.268421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] PATH_INFO: `/v3/users/c8af8f9a31c845f29f225141220fffbc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:37.269580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8af8f9a31c845f29f225141220fffbc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:37.271751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:37.535696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8af8f9a31c845f29f225141220fffbc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.536366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8af8f9a31c845f29f225141220fffbc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.557335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:37.558817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:37.572194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a391760-8607-4057-a6e3-4858e3626831 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:37.576982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:37.579651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:37.579651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:37.579651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] PATH_INFO: `/v3/users/6aa2dd17e6c14d079800f451cf3c8392` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:37.579651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa2dd17e6c14d079800f451cf3c8392)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:37.581377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:37.649179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8af8f9a31c845f29f225141220fffbc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.662284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8af8f9a31c845f29f225141220fffbc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.663011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8af8f9a31c845f29f225141220fffbc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.672123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:37.676564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1072/2139] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:37 2022] DELETE /identity/v3/users/c8af8f9a31c845f29f225141220fffbc => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:37.677383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:37.678798 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:37.768561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ae2ea-9c08-4ed2-9744-52c9fade0292 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:37.779652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:37.780568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:37.780970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:37.781329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] PATH_INFO: `/v3/projects/9da75e79415f49079b188a7077e06729` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:37.782703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da75e79415f49079b188a7077e06729)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:37.784830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:37.843547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa2dd17e6c14d079800f451cf3c8392'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.844238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa2dd17e6c14d079800f451cf3c8392'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.845559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa2dd17e6c14d079800f451cf3c8392'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.859440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa2dd17e6c14d079800f451cf3c8392'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.920695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da75e79415f49079b188a7077e06729', 'request_id': 'req-052aee98-bc7a-47d0-9884-76766a3f3792'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.927791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da75e79415f49079b188a7077e06729', 'request_id': 'req-052aee98-bc7a-47d0-9884-76766a3f3792'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.934598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da75e79415f49079b188a7077e06729', 'request_id': 'req-052aee98-bc7a-47d0-9884-76766a3f3792'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.935206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3e53e2ad-344e-47c7-8dd2-6197cc16293f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa2dd17e6c14d079800f451cf3c8392'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:37.948760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1068/2140] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:37 2022] DELETE /identity/v3/users/6aa2dd17e6c14d079800f451cf3c8392 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:37.951875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47159244-a04c-4b6b-a3d7-b0783cedee7e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:37.951875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47159244-a04c-4b6b-a3d7-b0783cedee7e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:37.951875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47159244-a04c-4b6b-a3d7-b0783cedee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:37.970471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-47159244-a04c-4b6b-a3d7-b0783cedee7e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:37.971689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-47159244-a04c-4b6b-a3d7-b0783cedee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.032065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1069/2141] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:38.033331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:38.033638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:38.033904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:38.056222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:38.057573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.065242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1070/2142] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:38.066010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:38.067254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.104537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] The token cache is being invalidate because project 9da75e79415f49079b188a7077e06729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:38.104916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.106030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.109005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1073/2143] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:37 2022] DELETE /identity/v3/projects/9da75e79415f49079b188a7077e06729 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:38.112331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:38.112331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.143048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d5340-9119-44cc-9122-e6a767175d04 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:38.150817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:38.151493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:38.151762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:38.152047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:38.154740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.171064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:38.172587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:38.184338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1071/2144] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:38.185578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:38.187256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.206006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a9019-b74d-4d0d-a91a-2e9345cf4739 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:38.215643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:38.215643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:38.215643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:38.216171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] PATH_INFO: `/v3/users/de84a1eca7064880880e9c0f2dda43d3` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:38.219918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de84a1eca7064880880e9c0f2dda43d3)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:38.222960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:38.260549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aee98-bc7a-47d0-9884-76766a3f3792 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:38.265296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:38.265842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:38.266238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:38.266435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:38.269085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:38.275989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.287753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:38.289901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:38.308186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1074/2145] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:38.486077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de84a1eca7064880880e9c0f2dda43d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.487003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de84a1eca7064880880e9c0f2dda43d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.488225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de84a1eca7064880880e9c0f2dda43d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.504236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de84a1eca7064880880e9c0f2dda43d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.602238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de84a1eca7064880880e9c0f2dda43d3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:38.625859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1072/2146] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:38 2022] DELETE /identity/v3/users/de84a1eca7064880880e9c0f2dda43d3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:38.929647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:38.933302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:38.986408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bf526-01c8-463b-b2cb-9d5a89f3ad0c service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.097607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.098338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.098659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.098945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] PATH_INFO: `/v3/projects/bd26d202c7b7438897ffd65805b64fd3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.100165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd26d202c7b7438897ffd65805b64fd3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.103018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:39.120997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:39.183606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:39.185363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:39.214986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753868-6496-4f45-9bc6-5ed1f9faf972 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.237062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd26d202c7b7438897ffd65805b64fd3', 'request_id': 'req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.245099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd26d202c7b7438897ffd65805b64fd3', 'request_id': 'req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.260620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd26d202c7b7438897ffd65805b64fd3', 'request_id': 'req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.304980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.305671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.306021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.306231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] PATH_INFO: `/v3/projects/deb29d922b58443cb7d2fc72a965ef82` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.307446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb29d922b58443cb7d2fc72a965ef82)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.309364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:39.369498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] The token cache is being invalidate because project bd26d202c7b7438897ffd65805b64fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:39.369858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.370666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.374445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1075/2147] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:38 2022] DELETE /identity/v3/projects/bd26d202c7b7438897ffd65805b64fd3 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:39.376482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:39.377864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:39.391619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53817143-64a3-4b2f-a0cf-d05ba9b2f0aa admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.396324 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.396972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.397260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.397539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] PATH_INFO: `/v3/users/2e62de83fcf547f2a5f01fafce26e5e0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.398761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e62de83fcf547f2a5f01fafce26e5e0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.406895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:39.457661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb29d922b58443cb7d2fc72a965ef82', 'request_id': 'req-26b6520c-0638-4bc3-b655-c69cb91d83ed'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.471345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb29d922b58443cb7d2fc72a965ef82', 'request_id': 'req-26b6520c-0638-4bc3-b655-c69cb91d83ed'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.572214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb29d922b58443cb7d2fc72a965ef82', 'request_id': 'req-26b6520c-0638-4bc3-b655-c69cb91d83ed'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.584445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] The token cache is being invalidate because project deb29d922b58443cb7d2fc72a965ef82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:39.584673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.585348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.587645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1073/2148] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:39 2022] DELETE /identity/v3/projects/deb29d922b58443cb7d2fc72a965ef82 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:39.591562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:39.595638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:39.612964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e62de83fcf547f2a5f01fafce26e5e0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.614090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e62de83fcf547f2a5f01fafce26e5e0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.666059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6520c-0638-4bc3-b655-c69cb91d83ed admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.674299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.675034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.675322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.675608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] PATH_INFO: `/v3/projects/9344d2690ed845f88d15ed0892a57250` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.676814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9344d2690ed845f88d15ed0892a57250)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.679083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:39.714773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e62de83fcf547f2a5f01fafce26e5e0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.728220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e62de83fcf547f2a5f01fafce26e5e0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.728950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e62de83fcf547f2a5f01fafce26e5e0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.741546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1076/2149] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:39 2022] DELETE /identity/v3/users/2e62de83fcf547f2a5f01fafce26e5e0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:39.744314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:39.754492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:39.823527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9344d2690ed845f88d15ed0892a57250', 'request_id': 'req-5639d785-ad57-437c-9ff5-af4984b09944'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.833037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9344d2690ed845f88d15ed0892a57250', 'request_id': 'req-5639d785-ad57-437c-9ff5-af4984b09944'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.851557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:39.858155 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd308-e13b-47de-bb2c-db8c34ca0241 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.918664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.919501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.919845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.920208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] PATH_INFO: `/v3/users/40605f70e0034578acbfc67f7b78b1d0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.921549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40605f70e0034578acbfc67f7b78b1d0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.926294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:39.951658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9344d2690ed845f88d15ed0892a57250', 'request_id': 'req-5639d785-ad57-437c-9ff5-af4984b09944'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.964421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] The token cache is being invalidate because project 9344d2690ed845f88d15ed0892a57250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:39.964728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.965417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:39.967242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1074/2150] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:39 2022] DELETE /identity/v3/projects/9344d2690ed845f88d15ed0892a57250 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:39.967794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:39.969145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:39.985391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d785-ad57-437c-9ff5-af4984b09944 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:39.991675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:39.992252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:39.992804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:39.993708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] PATH_INFO: `/v3/users/3bca1bf9118047769fc0ebef22a8e6aa` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:39.995042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bca1bf9118047769fc0ebef22a8e6aa)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:39.997262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.125266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40605f70e0034578acbfc67f7b78b1d0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.125949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40605f70e0034578acbfc67f7b78b1d0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.195274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40605f70e0034578acbfc67f7b78b1d0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.212842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bca1bf9118047769fc0ebef22a8e6aa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.213542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bca1bf9118047769fc0ebef22a8e6aa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.214447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bca1bf9118047769fc0ebef22a8e6aa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.215675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40605f70e0034578acbfc67f7b78b1d0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.216322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0bb03daa-d72c-4bba-9816-1848de1672c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40605f70e0034578acbfc67f7b78b1d0'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.227793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bca1bf9118047769fc0ebef22a8e6aa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.230860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1077/2151] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:39 2022] DELETE /identity/v3/users/40605f70e0034578acbfc67f7b78b1d0 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:40.232244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680338-e3dd-4960-89ee-573a93568ad3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.232485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680338-e3dd-4960-89ee-573a93568ad3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.232867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680338-e3dd-4960-89ee-573a93568ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.259215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-0e680338-e3dd-4960-89ee-573a93568ad3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:40.260456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0e680338-e3dd-4960-89ee-573a93568ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.320735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bca1bf9118047769fc0ebef22a8e6aa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.334239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1075/2152] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:39 2022] DELETE /identity/v3/users/3bca1bf9118047769fc0ebef22a8e6aa => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:40.335324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.336815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.354783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1078/2153] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:40.364562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.365103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.365642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.400478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:40.401979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.454063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee1bc3-6747-4f56-901a-9e1e5643cc74 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.469513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.470287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.470867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.471236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] PATH_INFO: `/v3/projects/77d68b6b47ea4ba6827e964c67307e76` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.473265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d68b6b47ea4ba6827e964c67307e76)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.475817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.491235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1079/2154] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:40.496609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.507840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525d49f-30d7-4dd8-adfa-f665fb7ab6fd None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.514059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.514863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.515215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.515564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.516397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:40.516803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.518447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.531341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1080/2155] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:40.535926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.547374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1e4f9-a724-4cb3-853b-f18f79572165 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.552286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.552962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.553242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.553546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.554220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.556442 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.617911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d68b6b47ea4ba6827e964c67307e76', 'request_id': 'req-0371d3e2-36c4-4c0f-9132-908f50b70a9b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.624003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d68b6b47ea4ba6827e964c67307e76', 'request_id': 'req-0371d3e2-36c4-4c0f-9132-908f50b70a9b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.715388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d68b6b47ea4ba6827e964c67307e76', 'request_id': 'req-0371d3e2-36c4-4c0f-9132-908f50b70a9b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.735193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1081/2156] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:40 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:40.735193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.736933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.744510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] The token cache is being invalidate because project 77d68b6b47ea4ba6827e964c67307e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:40.744900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.747836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:40.748144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1076/2157] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:40 2022] DELETE /identity/v3/projects/77d68b6b47ea4ba6827e964c67307e76 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:40.750460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.752213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.825808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371d3e2-36c4-4c0f-9132-908f50b70a9b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.829939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cc47-16f6-4acd-ab9f-8c0a2411e484 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.831215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.831953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.832345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.832726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.834501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.840087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.840292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.840387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.840387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.840387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] PATH_INFO: `/v3/projects/e993449a4d58431cada94ec21f5a13d9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.840387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e993449a4d58431cada94ec21f5a13d9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.841724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:40.855477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:40.937743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1077/2158] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:40 2022] POST /identity/v3/users => generated 283 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:40.937743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:40.937743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:40.949687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8556-c0b1-41f4-bd33-ab210c79efec admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:40.954452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:40.955207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:40.955343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:40.955627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] PATH_INFO: `/v3/users/ae82ad54ccc744b0beb4d9aebd97e336` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:40.956799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae82ad54ccc744b0beb4d9aebd97e336)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:40.958754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.069670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e993449a4d58431cada94ec21f5a13d9', 'request_id': 'req-cf73821c-2d08-4ff5-978b-e043e7f72753'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.076959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e993449a4d58431cada94ec21f5a13d9', 'request_id': 'req-cf73821c-2d08-4ff5-978b-e043e7f72753'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.082594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e993449a4d58431cada94ec21f5a13d9', 'request_id': 'req-cf73821c-2d08-4ff5-978b-e043e7f72753'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.184786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] The token cache is being invalidate because project e993449a4d58431cada94ec21f5a13d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:41.185496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.185999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.191427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1082/2159] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:40 2022] DELETE /identity/v3/projects/e993449a4d58431cada94ec21f5a13d9 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:41.191427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.191427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.195301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae82ad54ccc744b0beb4d9aebd97e336'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.197306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae82ad54ccc744b0beb4d9aebd97e336'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.197958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae82ad54ccc744b0beb4d9aebd97e336'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.210551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae82ad54ccc744b0beb4d9aebd97e336'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.255653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73821c-2d08-4ff5-978b-e043e7f72753 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.260987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.261624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.261909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.262188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] PATH_INFO: `/v3/projects/518ea4f80df4498d8d6e059f113defa4` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.263385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518ea4f80df4498d8d6e059f113defa4)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.265310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.283185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae82ad54ccc744b0beb4d9aebd97e336'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.296267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1078/2160] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:40 2022] DELETE /identity/v3/users/ae82ad54ccc744b0beb4d9aebd97e336 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:41.297086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.299605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.364645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cecf-0922-4488-80e8-134a65102462 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.370265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.370926 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.371214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.371503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.372209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.373903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f200a-0db7-4e5c-ac4a-67f6ea996569 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.385162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1079/2161] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:41.386713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.387028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.387310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.407461 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:41.408689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.417644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1080/2162] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:41.419975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.429391 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442220db-d8ca-4237-b84d-021ca74bc1e3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.442582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518ea4f80df4498d8d6e059f113defa4', 'request_id': 'req-fe018e66-0363-4dd1-b85f-329399178419'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.449670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518ea4f80df4498d8d6e059f113defa4', 'request_id': 'req-fe018e66-0363-4dd1-b85f-329399178419'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.450010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:41.455395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518ea4f80df4498d8d6e059f113defa4', 'request_id': 'req-fe018e66-0363-4dd1-b85f-329399178419'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.488725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.489355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.489645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.489955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.490743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.492592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.504129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1081/2163] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:41.504129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.509589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.525287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726c11-e37d-4bfc-8f6b-64db0c758706 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.537633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.539415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.539415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.539415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] PATH_INFO: `/v3/users/8835b71fe1324373bfd00e30114c5d0b` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.541275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8835b71fe1324373bfd00e30114c5d0b)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.543354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.549308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] The token cache is being invalidate because project 518ea4f80df4498d8d6e059f113defa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:41.549731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.551146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-fe018e66-0363-4dd1-b85f-329399178419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.553274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1083/2164] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:41 2022] DELETE /identity/v3/projects/518ea4f80df4498d8d6e059f113defa4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:41.556446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.556927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.557488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.583822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:41.585106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.593926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1084/2165] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:41.594846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.598988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.612947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d35a61-963c-437a-8d9c-9bcd8fb2e620 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.618543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.619534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.619797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.620089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] PATH_INFO: `/v3/projects/cd50183558114776b1db5fd3ee4ee7cf/users/16e6308e10cd46eea31c48a02c0a9e37/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.636032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd50183558114776b1db5fd3ee4ee7cf, user_id=16e6308e10cd46eea31c48a02c0a9e37, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.642525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f82fbf-d68e-435e-83ce-80a4b4513177 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.695550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8835b71fe1324373bfd00e30114c5d0b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.696747 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8835b71fe1324373bfd00e30114c5d0b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.697395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8835b71fe1324373bfd00e30114c5d0b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.710106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8835b71fe1324373bfd00e30114c5d0b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.715301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1085/2166] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:41 2022] PUT /identity/v3/projects/cd50183558114776b1db5fd3ee4ee7cf/users/16e6308e10cd46eea31c48a02c0a9e37/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:41.717179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.717179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.717179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.736915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:41.738273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.810075 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1086/2167] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:41.818970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.819122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e23e2a70-f3cc-47bc-b0da-56c539d2dea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8835b71fe1324373bfd00e30114c5d0b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:41.819539 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.826544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1082/2168] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:41 2022] DELETE /identity/v3/users/8835b71fe1324373bfd00e30114c5d0b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:41.828274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccbf3-9a40-48e6-a8e7-0a709a5babfd None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.828795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccbf3-9a40-48e6-a8e7-0a709a5babfd None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.829291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccbf3-9a40-48e6-a8e7-0a709a5babfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.833796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82560dd5-77ba-4c13-84ea-d9329a76abbb None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.866013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5c6ccbf3-9a40-48e6-a8e7-0a709a5babfd None None] MFA Rules not processed for user `16e6308e10cd46eea31c48a02c0a9e37`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:41.869017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5c6ccbf3-9a40-48e6-a8e7-0a709a5babfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.909574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.910654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.911046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.911428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.912243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:41.912655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.914400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.928053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1087/2169] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:41.929123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:41.932981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.946608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da900808-d386-47cf-bdd3-a62092cc648a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:41.951704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:41.952358 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.952645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.953076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] PATH_INFO: `/v3/projects/aa16f0552e824785b660fa0ef991a0e0` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.954266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa16f0552e824785b660fa0ef991a0e0)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:41.956330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:41.958676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1083/2170] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:41.963863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6eed-48a6-4fce-a93b-17611f647715 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.964160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6eed-48a6-4fce-a93b-17611f647715 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.964443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6eed-48a6-4fce-a93b-17611f647715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:41.985091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-39ed6eed-48a6-4fce-a93b-17611f647715 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:41.986336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-39ed6eed-48a6-4fce-a93b-17611f647715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:41.995150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1084/2171] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:41.996398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:41.996910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:41.998690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.021499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:42.021499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.028891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1085/2172] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:42.029724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.040691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207891b8-dec2-4e40-9c79-ca50c6f6ec20 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.045803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.046515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.046803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.047083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.047757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.049740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.064217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa16f0552e824785b660fa0ef991a0e0', 'request_id': 'req-660b58ce-f625-4a2f-8922-b36b7d2adc6d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:42.071280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa16f0552e824785b660fa0ef991a0e0', 'request_id': 'req-660b58ce-f625-4a2f-8922-b36b7d2adc6d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:42.075023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:42.078557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa16f0552e824785b660fa0ef991a0e0', 'request_id': 'req-660b58ce-f625-4a2f-8922-b36b7d2adc6d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:42.170217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] The token cache is being invalidate because project aa16f0552e824785b660fa0ef991a0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:42.170572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:42.171206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:42.172646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1086/2173] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.176005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1088/2174] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:41 2022] DELETE /identity/v3/projects/aa16f0552e824785b660fa0ef991a0e0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:42.176089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.176163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.176163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.177794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.244719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794239db-4fff-4c62-8b1a-aed4978ca1f8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.246771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b58ce-f625-4a2f-8922-b36b7d2adc6d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.249663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.249814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.250467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.250651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.250784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.250892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.251014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.251159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.251730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:42.251838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.252045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.253670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.253801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.269667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1089/2175] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:42.271106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.272697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.338405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61c55f-119b-4e02-836f-dc108ec7cdb0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.342208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.342880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.343176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.343444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.345425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.354558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1087/2176] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.354558 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.356821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.365398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.366911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca377e5-0453-480f-b1a3-d4de2d7e1b42 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.382278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1090/2177] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:42.384873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.385374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.385793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.415913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:42.417609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.453574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c1bc-3e1a-4fba-a80d-fe7696f3f370 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.460963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.461228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.461630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.462579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.463265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.465259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.478232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:42.506164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1091/2178] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:42.507146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.518463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8698a-6f7b-4e2f-b218-81616056706a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.523609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.525471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.525784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.526064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.526815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.528980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3886f-7014-48f7-b24c-e0f4315ebdea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.591155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1088/2179] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/users => generated 282 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.592056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.605611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfeb29a-739d-4816-96c6-d3ea12afa8d3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.611850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.612512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.612867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.613099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.615749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.615928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1092/2180] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.616059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.617669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.618583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.618877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.646093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:42.646093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:42.719385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1093/2181] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:42.720686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.727042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1089/2182] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.728140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.732902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b243a5-546f-46d5-9c65-8015f904d755 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.739653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.740447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.741247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.741675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.743369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.749339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.749465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c3e55-3726-426a-94fd-13c4d1735b5b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.755764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.756751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.757101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.757826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.758197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.760438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1094/2183] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:42.761573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.763709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.781304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f497b-fab6-4231-86f6-bfbd4e1aaf9a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.786418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.787048 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.787335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.787619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.788335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:42.788647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.790538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.802271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:42.806114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1095/2184] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:42.806994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.820044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a90c-cf1a-412e-a2ce-4004dadad09a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.829524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.830199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.833287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.833287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.833287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.833287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.850547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1096/2185] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:42.851459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.861992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babb71-6300-4c92-84df-0203afecf16d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.870108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.871172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.871564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.872576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.873521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.875226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.877840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1090/2186] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:42.878763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.889454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5791d3-0a56-4a72-9b10-f505c3fedc69 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.892454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1097/2187] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:42.893483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:42.903380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.904019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.904471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.904764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.905522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.908469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:42.909399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13eebe-8120-4f8c-8876-66c83e35436d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:42.915929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:42.917364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:42.917852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:42.918311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] PATH_INFO: `/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd/users/9f80b67312e048e195a20df202a28b8a/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:42.941385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d0a093fe0949ffae320b1df0b4b2cd, user_id=9f80b67312e048e195a20df202a28b8a, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:42.945384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.028002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1098/2188] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:42 2022] PUT /identity/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd/users/9f80b67312e048e195a20df202a28b8a/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:43.028987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1091/2189] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:42 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:43.030120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.033136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.099851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a62c1-43df-47c5-b75e-b7e0be5ba9b7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.105562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.106298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.112476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.112476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] PATH_INFO: `/v3/projects/e81a1209771a4796807a8e439a27618b/users/084fa3cc947d4738b6993fcad880031e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.112817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a083e-3f43-49d8-8af2-06bbce7cf47d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.117180 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.118217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.118535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.118819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.119532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.121221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.128517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a1209771a4796807a8e439a27618b, user_id=084fa3cc947d4738b6993fcad880031e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.131327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.132400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1099/2190] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:43.133071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.137105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.149712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:43.151033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8092005-3df4-4c9c-b58f-18ccc5eb6f4d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.155905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.156696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.156988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.157271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] PATH_INFO: `/v3/projects/59b0a55a29a24b0ca15868fb25a73ca4` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.158488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b0a55a29a24b0ca15868fb25a73ca4)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.160417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.206411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1092/2191] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:43 2022] PUT /identity/v3/projects/e81a1209771a4796807a8e439a27618b/users/084fa3cc947d4738b6993fcad880031e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:43.207411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.275439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa568d9-643b-42c2-89d2-b3dd826ea8bb admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.280225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.280800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.281071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.281352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.282049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.283682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.294187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1093/2192] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:43.294999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.304518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ba957-2471-4cf8-bb07-be676e87f6af admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.310201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.310841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.311196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.311434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.312131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.314084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.323600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b0a55a29a24b0ca15868fb25a73ca4', 'request_id': 'req-00899014-a4dc-4c31-b01e-6e26a0e4240f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.331941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b0a55a29a24b0ca15868fb25a73ca4', 'request_id': 'req-00899014-a4dc-4c31-b01e-6e26a0e4240f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.338087 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b0a55a29a24b0ca15868fb25a73ca4', 'request_id': 'req-00899014-a4dc-4c31-b01e-6e26a0e4240f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.403895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1094/2193] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:43 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:43.403895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.412174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448afac-afb9-452a-9d97-846e0ac9e256 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.430914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] The token cache is being invalidate because project 59b0a55a29a24b0ca15868fb25a73ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:43.431169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.431857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-00899014-a4dc-4c31-b01e-6e26a0e4240f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.433745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1100/2194] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:43 2022] DELETE /identity/v3/projects/59b0a55a29a24b0ca15868fb25a73ca4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:43.435187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.435471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.435744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.470936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] MFA Rules not processed for user `084fa3cc947d4738b6993fcad880031e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:43.472390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.478821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.479529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.479942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.480230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.480938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.482669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.494036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1095/2195] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:43.495336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.499235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.512691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9c71-f5f8-4351-b67e-077e07219dc3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.517543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.518215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.518534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.518819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] PATH_INFO: `/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd/users/9f80b67312e048e195a20df202a28b8a/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.520819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d0a093fe0949ffae320b1df0b4b2cd, user_id=9f80b67312e048e195a20df202a28b8a, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.523894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.536650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1096/2196] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:43 2022] PUT /identity/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd/users/9f80b67312e048e195a20df202a28b8a/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:43.537436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.547877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.560663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1101/2197] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:43.562719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.563999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.581642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2138-2501-4131-94b1-a7171bf6e576 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.622123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:43.632370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9baf84-7d79-4630-b88c-b981040dd740 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.639074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.639989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.640102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.640300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.641054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.642829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.659037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1097/2198] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:43.665370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.665370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.665704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.665704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.665704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.665704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] PATH_INFO: `/v3/users/4aa59ff023ed4386a469d503d0fb239e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.666054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa59ff023ed4386a469d503d0fb239e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.667948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.676052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a83cb-0943-48b3-a05a-e7470e3e40d4 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.683053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.683904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.684287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.684574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] PATH_INFO: `/v3/projects/5df2cf30f5194eb58881a670ba990acd/users/c91083376132452693a47d709fe85ef7/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.701147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df2cf30f5194eb58881a670ba990acd, user_id=c91083376132452693a47d709fe85ef7, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.703917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7442b2-ee03-4c68-993a-0e3553b0fa8f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:43.795306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1098/2199] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:43 2022] PUT /identity/v3/projects/5df2cf30f5194eb58881a670ba990acd/users/c91083376132452693a47d709fe85ef7/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:43.796738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.797032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.797454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.827287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] MFA Rules not processed for user `9f80b67312e048e195a20df202a28b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:43.828519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.898335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1099/2200] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:43.899307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:43.901743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:43.946159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa59ff023ed4386a469d503d0fb239e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.947119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa59ff023ed4386a469d503d0fb239e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:43.970002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749e4f5-d951-445b-8a3a-1c4f3c77f474 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:43.974723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:43.975401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:43.975724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:43.975936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:43.976639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:43.979445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575451d-47bf-4f7d-b135-7b837519862a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.005612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa59ff023ed4386a469d503d0fb239e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.018473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa59ff023ed4386a469d503d0fb239e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.019119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa59ff023ed4386a469d503d0fb239e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.028808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:44.031863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1102/2201] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:43 2022] DELETE /identity/v3/users/4aa59ff023ed4386a469d503d0fb239e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:44.035233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.035683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.096631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1100/2202] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:43 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:44.099122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02578e6-8880-4391-954b-57b90404be72 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.099604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02578e6-8880-4391-954b-57b90404be72 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.100161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02578e6-8880-4391-954b-57b90404be72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.118114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2896c-f03c-45d7-b610-83e4d1dc71ff admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.148463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e02578e6-8880-4391-954b-57b90404be72 None None] MFA Rules not processed for user `c91083376132452693a47d709fe85ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:44.150223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e02578e6-8880-4391-954b-57b90404be72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.183588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.184223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.184506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.184835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.185551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.187256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.198992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1103/2203] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:44.198992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.198992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.226017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1101/2204] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:44.226017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02578e6-8880-4391-954b-57b90404be72 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.226017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e02578e6-8880-4391-954b-57b90404be72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.247821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02578e6-8880-4391-954b-57b90404be72 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.253060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.253707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.253988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.254268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.255038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.256932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.275090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a606-8fdf-4e3f-8266-4886607003ec admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.279152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.279781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.280072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.280361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.282250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.290154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:44.349581 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.357507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.361510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1104/2205] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:44.362444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.374307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9e53a-4bfe-449d-af6d-67de3a3e569f service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.385215 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1102/2206] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:44 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:44.386093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.389280 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.399367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.399367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.399367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] PATH_INFO: `/v3/projects/062a2e97253143909a598ff41f5af5e0/users/ccc842db6bee45069de11f4d3acc6073/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.399799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.406522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79c3b1-6fa7-4d8f-b7e7-0d7f8c8b8a74 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.411771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=062a2e97253143909a598ff41f5af5e0, user_id=ccc842db6bee45069de11f4d3acc6073, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.415405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.419189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.420050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.420567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.420851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.421639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.423645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.542327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1105/2207] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:44 2022] PUT /identity/v3/projects/062a2e97253143909a598ff41f5af5e0/users/ccc842db6bee45069de11f4d3acc6073/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:44.543388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.547671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1103/2208] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:44 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:44.547671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.550268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.578621 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:44.611364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffe01b-19ef-45a2-ab02-1494b32d73c1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.637984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fedc5c-a3aa-40a4-bb34-22605c637773 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.642827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.643476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.643760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.644051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] PATH_INFO: `/v3/projects/63715cfd5dd64054a90a060d74f580bb` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.645244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=63715cfd5dd64054a90a060d74f580bb)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.647194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.672513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.673148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.673437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.673718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.674475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.676233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.691204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1106/2209] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:44.691204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.697749 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1382-e843-495f-a963-0071e2c43120 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.703766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.704566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.704900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.705385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.706088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:44.708607 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63915c38-74da-4046-ae72-237f124c94ed admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:44.768492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63715cfd5dd64054a90a060d74f580bb', 'request_id': 'req-e45fb165-b80b-43be-98ae-d62cf2a093fc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.774149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63715cfd5dd64054a90a060d74f580bb', 'request_id': 'req-e45fb165-b80b-43be-98ae-d62cf2a093fc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.783251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:44.846474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1107/2210] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:44 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:44.847563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.847863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.848147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.867110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63715cfd5dd64054a90a060d74f580bb', 'request_id': 'req-e45fb165-b80b-43be-98ae-d62cf2a093fc'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.871066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:44.881873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] MFA Rules not processed for user `ccc842db6bee45069de11f4d3acc6073`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:44.882585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] The token cache is being invalidate because project 63715cfd5dd64054a90a060d74f580bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:44.882972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.883139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.883859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:44.886382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1104/2211] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:44 2022] DELETE /identity/v3/projects/63715cfd5dd64054a90a060d74f580bb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:44.887213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.888666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.951055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fb165-b80b-43be-98ae-d62cf2a093fc admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:44.955145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:44.955771 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:44.956088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:44.960338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:44.960475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1108/2212] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:44.960475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:44.960475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:44.968632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.015665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:45.030009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0465277-aa46-415b-8775-7ed8d955d8c7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.031625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.033178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.035322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.036030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.036314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.036597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] PATH_INFO: `/v3/projects/76929436f9ba47dea7bd1d1539c594f6/users/4d7fe195d3ba4d4880fb8b95a2dda500/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.043352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1105/2213] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:45.044290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.046105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:45.047609 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.054786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76929436f9ba47dea7bd1d1539c594f6, user_id=4d7fe195d3ba4d4880fb8b95a2dda500, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.056743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.064704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb8ba1-b017-47ec-8f22-9fdb9ea1453f service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.069847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.070513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.070802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.071121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.072550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.074455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6487a-cb12-48a1-8f07-86ca5ec44159 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.084670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1106/2214] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:45.086038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.086273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.086577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.107353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:45.108570 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.117063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1107/2215] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:45.118201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.122576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.132407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1109/2216] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:44 2022] PUT /identity/v3/projects/76929436f9ba47dea7bd1d1539c594f6/users/4d7fe195d3ba4d4880fb8b95a2dda500/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:45.133487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.135574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b92a2-1847-4a8d-834a-3a7231bb6af8 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.191772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.192398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.192723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.193042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.199866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725170e-41fc-4e40-9cac-defa51ae830c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.204934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.205614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.205903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.206185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] PATH_INFO: `/v3/projects/e5a859a37406451fb33f2c716a944ffb/users/81397ee6bd284cceba99679b57c4a843/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.224003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a859a37406451fb33f2c716a944ffb, user_id=81397ee6bd284cceba99679b57c4a843, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.226742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.253381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.254880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759d33-0a3b-4bdd-bb70-b3fd9e258ed5 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.264525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1108/2217] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:45.265729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.266220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.266529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.287243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:45.288582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.297737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1109/2218] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:45.298645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.308676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bb2f4-5feb-4326-81c9-779d3bc0e12a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.313724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.314341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.314680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.314962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.315667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.317252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.327678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1110/2219] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:45 2022] PUT /identity/v3/projects/e5a859a37406451fb33f2c716a944ffb/users/81397ee6bd284cceba99679b57c4a843/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:45.329114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.330595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1110/2220] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:45.332464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.345237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618823-24a9-4510-8d9a-6b679a1c5c2e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.408894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31aee8-959d-441f-8bb7-8d15140d3bf8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.414250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.414655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.415009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.415997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.416133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.416345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.416459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.416913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.417083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:45.417536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.418163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.419413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.420386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:45.422499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.432700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1111/2221] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:45.433544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.436949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1111/2222] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:45.437496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.444179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d659017-0f76-4426-8127-6cae6ae7ab36 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.448268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf5822-1681-4487-9139-bf975a066b43 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.449735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.450428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.450722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.451003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] PATH_INFO: `/v3/projects/76929436f9ba47dea7bd1d1539c594f6/users/4d7fe195d3ba4d4880fb8b95a2dda500/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.452956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76929436f9ba47dea7bd1d1539c594f6, user_id=4d7fe195d3ba4d4880fb8b95a2dda500, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.453182 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.453929 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.454221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.454530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.455276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.455778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.457236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f08d9-b51e-40b2-8100-52ef3b513b04 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.468816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1112/2223] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:45 2022] PUT /identity/v3/projects/76929436f9ba47dea7bd1d1539c594f6/users/4d7fe195d3ba4d4880fb8b95a2dda500/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:45.469640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.505165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:45.547974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8035a-cf6d-4000-b1cc-09dcece80e00 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.552905 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.553582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.553865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.554140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] PATH_INFO: `/v3/projects/e5a859a37406451fb33f2c716a944ffb/users/81397ee6bd284cceba99679b57c4a843/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.556261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a859a37406451fb33f2c716a944ffb, user_id=81397ee6bd284cceba99679b57c4a843, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.558970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.565033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1112/2224] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:45.566375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084789e0-0932-4b5f-ab98-7ddb71fec2a6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.580006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084789e0-0932-4b5f-ab98-7ddb71fec2a6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.580006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084789e0-0932-4b5f-ab98-7ddb71fec2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.580220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1113/2225] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:45 2022] PUT /identity/v3/projects/e5a859a37406451fb33f2c716a944ffb/users/81397ee6bd284cceba99679b57c4a843/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:45.580220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.599213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-084789e0-0932-4b5f-ab98-7ddb71fec2a6 None None] MFA Rules not processed for user `4d7fe195d3ba4d4880fb8b95a2dda500`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:45.606383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-084789e0-0932-4b5f-ab98-7ddb71fec2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.654783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae998-78c2-467d-9679-7b48889b3f05 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.678766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1113/2226] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:45.680272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.680562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.682514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.716759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] MFA Rules not processed for user `81397ee6bd284cceba99679b57c4a843`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:45.718004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:45.720305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.721765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.722053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.722383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.723114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.725185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.791903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1114/2227] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:45.792901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.844947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1114/2228] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:45 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:45.848402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.853937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23965a-5eef-4aaa-ae3c-dac2fde9a92e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.857585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.857984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd223c-6117-48f8-bf27-e1a8099609bb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.858203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.858530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.858812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.862919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.863629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.863927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.864226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.864923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.869776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.869889 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.870852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.881194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1115/2229] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:45.882223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.883642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1115/2230] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:45.893525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d988-b14e-4b72-8b71-81c05936eefb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.899198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.899320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.899671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.899890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.900663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.902286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:45.914593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1116/2231] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:45.918665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:45.929343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc035efd-a215-4850-a1fb-651f1573f633 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:45.934543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:45.935206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:45.935496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:45.935839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] PATH_INFO: `/v3/projects/834ec9d4361246999cc4e822acaa6b77/users/6c67d6fcda3f4a938b2728f7ce2fc3ea/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:45.954196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834ec9d4361246999cc4e822acaa6b77, user_id=6c67d6fcda3f4a938b2728f7ce2fc3ea, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:45.961454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b19efb-61d7-44ea-b36f-d963c38c9974 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:46.000416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:46.011481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822a249-d038-458d-a213-f7369afea47c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:46.015795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:46.022169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:46.022169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:46.022169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:46.040852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:46.042414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:46.052279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1116/2232] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:45 2022] PUT /identity/v3/projects/834ec9d4361246999cc4e822acaa6b77/users/6c67d6fcda3f4a938b2728f7ce2fc3ea/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:46.057611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:46.058112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:46.058404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1117/2233] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:46.065760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:46.116786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:46.116786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:46.205143 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1117/2234] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:46.215946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:46.217747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:46.280416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9bd8d-a820-4dc1-9869-3e9d1a6b2890 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:46.334413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:46.335045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:46.335337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:46.335619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:46.346088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:46.347932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b69fc-5dc0-4d8a-b94f-c98f8b350ba3 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:46.360534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1118/2235] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:46.449368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:46.541288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:46.552008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:46.620691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca537fd6-ce1a-4d28-9a9d-cc97187b652a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:46.624815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:46.625452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:46.625764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:46.626047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:46.627880 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:46.689934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:46.691539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:46.708515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1118/2236] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:46.983550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:47.143554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:47.815156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:47.894211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:47.895137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:47.895669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:47.934681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] MFA Rules not processed for user `6c67d6fcda3f4a938b2728f7ce2fc3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:47.934681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:48.015360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1119/2237] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:48.027276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:48.036101 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336a5eb-83ab-4d1c-b7a7-f7c479cf4e26 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:48.039830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:48.041157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:48.041437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:48.041714 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:48.050647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:48.052983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf8da3-cf45-4c69-80f7-a3ac3f85d66e service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:48.064682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:48.066599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1119/2238] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:48.076841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc456c5-08fd-4b1a-a94e-34436eaead9d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:48.081622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:48.082272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:48.082587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:48.082872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] PATH_INFO: `/v3/users/084fa3cc947d4738b6993fcad880031e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:48.084089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084fa3cc947d4738b6993fcad880031e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:48.086028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:48.183559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:48.288320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084fa3cc947d4738b6993fcad880031e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:48.289056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084fa3cc947d4738b6993fcad880031e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:48.411447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084fa3cc947d4738b6993fcad880031e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:48.424266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084fa3cc947d4738b6993fcad880031e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:48.424977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084fa3cc947d4738b6993fcad880031e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:48.437388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1120/2239] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:48 2022] DELETE /identity/v3/users/084fa3cc947d4738b6993fcad880031e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:48.442763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:48.443387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:48.443789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:48.465567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:48.467007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:48.536953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1120/2240] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:48.546447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:48.548041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:48.609210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c2803-32dd-486d-bb8b-bd88655191b1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:48.612826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:48.613439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:48.613723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:48.614000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:48.623502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:48.625144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:48.638319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1121/2241] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:49.166109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:49.167956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:49.183287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b66ba-8ff2-449a-9451-bf5a01ac1387 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:49.188774 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:49.189492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:49.190030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:49.190540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] PATH_INFO: `/v3/projects/e81a1209771a4796807a8e439a27618b` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:49.192122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81a1209771a4796807a8e439a27618b)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:49.195262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:49.365622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81a1209771a4796807a8e439a27618b', 'request_id': 'req-a1b94263-d7eb-479a-a9e3-200df2425afa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.373274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81a1209771a4796807a8e439a27618b', 'request_id': 'req-a1b94263-d7eb-479a-a9e3-200df2425afa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.377320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:49.465776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81a1209771a4796807a8e439a27618b', 'request_id': 'req-a1b94263-d7eb-479a-a9e3-200df2425afa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.477407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] The token cache is being invalidate because project e81a1209771a4796807a8e439a27618b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:49.478293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.478979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.480509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1121/2242] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:49 2022] DELETE /identity/v3/projects/e81a1209771a4796807a8e439a27618b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:49.484876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:49.495900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ee49-3ee9-4387-8e92-9b79c810bdba service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:49.553645 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:49.554300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:49.554602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:49.554951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] PATH_INFO: `/v3/users/4d7fe195d3ba4d4880fb8b95a2dda500` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:49.557854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7fe195d3ba4d4880fb8b95a2dda500)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:49.559854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:49.715179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:49.786601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:49.788009 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:49.801016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b94263-d7eb-479a-a9e3-200df2425afa admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:49.805986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:49.806765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:49.807053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:49.807344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] PATH_INFO: `/v3/users/ccc842db6bee45069de11f4d3acc6073` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:49.808518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc842db6bee45069de11f4d3acc6073)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:49.810561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:49.841153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7fe195d3ba4d4880fb8b95a2dda500'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.841842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7fe195d3ba4d4880fb8b95a2dda500'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.940198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7fe195d3ba4d4880fb8b95a2dda500'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.953961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7fe195d3ba4d4880fb8b95a2dda500'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.954676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2c605dab-c6b3-4894-befb-95f63a9d7c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7fe195d3ba4d4880fb8b95a2dda500'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:49.968725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1122/2243] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:49 2022] DELETE /identity/v3/users/4d7fe195d3ba4d4880fb8b95a2dda500 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:50.158003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc842db6bee45069de11f4d3acc6073'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.159115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc842db6bee45069de11f4d3acc6073'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.159935 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc842db6bee45069de11f4d3acc6073'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.176565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc842db6bee45069de11f4d3acc6073'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.256783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc842db6bee45069de11f4d3acc6073'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.277987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.278284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.278587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.281756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1122/2244] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:49 2022] DELETE /identity/v3/users/ccc842db6bee45069de11f4d3acc6073 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:50.300874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:50.302133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.367808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1123/2245] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:50.378281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.379987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.411682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.413134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.425991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb28a3-c19b-4a4c-b3cd-80d6dfc54f9e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:50.442710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275cb2c-f57a-420a-9f8e-d30c446c5295 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:50.444453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:50.445231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.445502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.445795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] PATH_INFO: `/v3/projects/76929436f9ba47dea7bd1d1539c594f6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.447315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:50.447818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76929436f9ba47dea7bd1d1539c594f6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:50.447927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.448132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.448413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.450049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:50.458515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:50.460155 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997a59-110a-48b0-adde-6e5aa207246e service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:50.470818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1123/2246] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:50.585930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76929436f9ba47dea7bd1d1539c594f6', 'request_id': 'req-6036572b-4d57-4c44-b8be-d9c1663cf80f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.592585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76929436f9ba47dea7bd1d1539c594f6', 'request_id': 'req-6036572b-4d57-4c44-b8be-d9c1663cf80f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.597848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76929436f9ba47dea7bd1d1539c594f6', 'request_id': 'req-6036572b-4d57-4c44-b8be-d9c1663cf80f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.702959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] The token cache is being invalidate because project 76929436f9ba47dea7bd1d1539c594f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:50.703174 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.703809 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6036572b-4d57-4c44-b8be-d9c1663cf80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:50.705585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1124/2247] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:50 2022] DELETE /identity/v3/projects/76929436f9ba47dea7bd1d1539c594f6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:50.710339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed605-22b5-4aba-ba5d-3bd8611798dc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.710663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed605-22b5-4aba-ba5d-3bd8611798dc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.710951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ed605-22b5-4aba-ba5d-3bd8611798dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.733296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7b1ed605-22b5-4aba-ba5d-3bd8611798dc None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:50.734550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7b1ed605-22b5-4aba-ba5d-3bd8611798dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.813208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1124/2248] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:50.821608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.822105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.822590 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.866482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.866482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.866482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.871979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:50.873368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.889058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] MFA Rules not processed for user `6c67d6fcda3f4a938b2728f7ce2fc3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:50.889058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.894737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1125/2249] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:50.898246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.914063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ecd4f-60f5-4670-a50f-56a703c66371 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:50.927778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:50.928625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.928943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.929233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.929964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:50.930265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:50.932800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:50.945641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1126/2250] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:03:50.950617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.956807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1125/2251] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:50.963186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038522f0-73a8-4ffd-88f8-1784efa7028a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:50.968521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:50.969154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.969441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.969723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.970420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:50.973513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:50.996989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.998711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.021186 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207d03-b47d-4b1a-b699-ac22585a5f2c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.035140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.035896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.036251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.036636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] PATH_INFO: `/v3/projects/062a2e97253143909a598ff41f5af5e0` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.037936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=062a2e97253143909a598ff41f5af5e0)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.040191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.081234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1127/2252] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:51.085471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.155685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33046cdc-a39b-48b5-bdae-6dd2451d58f0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.164001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.164662 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.169038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.169038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.169038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.169038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.182244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062a2e97253143909a598ff41f5af5e0', 'request_id': 'req-e6912e7f-6872-4fa1-ba76-f964a1122394'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:51.188573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062a2e97253143909a598ff41f5af5e0', 'request_id': 'req-e6912e7f-6872-4fa1-ba76-f964a1122394'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:51.247266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1128/2253] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:51.251051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.262402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c95735-517b-4948-9851-d2ad67917697 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.274163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062a2e97253143909a598ff41f5af5e0', 'request_id': 'req-e6912e7f-6872-4fa1-ba76-f964a1122394'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:51.278076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:51.286825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] The token cache is being invalidate because project 062a2e97253143909a598ff41f5af5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:03:51.287124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:51.287802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6912e7f-6872-4fa1-ba76-f964a1122394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:51.290701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1126/2254] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:50 2022] DELETE /identity/v3/projects/062a2e97253143909a598ff41f5af5e0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:51.296634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b8f5-57db-4a6e-ad93-eeb79f4efad7 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.297152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b8f5-57db-4a6e-ad93-eeb79f4efad7 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.297608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b8f5-57db-4a6e-ad93-eeb79f4efad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.321130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e6a7b8f5-57db-4a6e-ad93-eeb79f4efad7 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:51.322398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e6a7b8f5-57db-4a6e-ad93-eeb79f4efad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.335709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.336341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.336649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.337033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.337842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.339526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.354381 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1129/2255] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:51.355728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.358040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.372593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deacc-6a21-4c39-a03f-b1325ee5d90d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.377665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.378498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.379016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.379312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.380097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.381821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.390327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1127/2256] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:51.395558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1130/2257] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:51.399338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.399629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.400788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.404003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.416066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec816bd-8c3f-476f-9a46-fa28370d661a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.421063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:51.422465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.423394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:51.429531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.429531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.429531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] PATH_INFO: `/v3/projects/9c7e51fb281446fdad1e82543789b6e9/users/3ffc1548791e4762adea85a390764cf0/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.429818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.433876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1128/2258] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:51.438173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.441602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7e51fb281446fdad1e82543789b6e9, user_id=3ffc1548791e4762adea85a390764cf0, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.444482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735acbe-d01c-4355-8988-e3dc805c7d34 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.448394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df2447-10de-4266-8309-c2f1006bbb14 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.453716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.454389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.454655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.454937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.461711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:03:51.461711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.461711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.469578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1129/2259] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:03:51.473905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.485180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733f9ea-541e-4c2e-9ce3-8f42d82546af admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.490995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.491870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.493616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.493911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.494611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.496811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.507448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1131/2260] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:51 2022] PUT /identity/v3/projects/9c7e51fb281446fdad1e82543789b6e9/users/3ffc1548791e4762adea85a390764cf0/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:51.518748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.519067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.519368 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.550068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] MFA Rules not processed for user `3ffc1548791e4762adea85a390764cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:51.551420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.615719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1130/2261] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:51.617218 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.620076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1132/2262] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:51.620976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.692047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a98a9-abce-4c89-acf3-18e695034c3b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.713855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350295c0-fdbd-4994-bc8c-6d5cca54cead admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.719487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.720241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.720693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.721074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.721891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.724657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.763271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.763271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.763271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.763271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.763672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.765584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.817492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1131/2263] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:51.821376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.832038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a93e0-2104-45af-bc95-46b2c0bd12b7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.838188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.838988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.839354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.839711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.840499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.842294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.855043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1132/2264] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:51.858897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.869366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb64d5d-3098-4d79-b83b-be5bf7703ad2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.874496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.875109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.875403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.875718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.876417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.878094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.881160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1133/2265] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/projects => generated 365 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:03:51.886826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.889486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1133/2266] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:51.894863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:51.907135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22647af0-dd8a-4376-ab44-038b4e8abe23 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.957757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2d9c2-487b-456b-ad2a-2f213d232583 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.974739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.974739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.974739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.974739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.975096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.975096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2350b-8b11-478c-93cd-7845af7984e3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.992397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.993219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.993512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.993797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] PATH_INFO: `/v3/projects/39115160ca364b2590af12fccde98256/users/78888cc145ba4052be05b1c321de44fd/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.009475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39115160ca364b2590af12fccde98256, user_id=78888cc145ba4052be05b1c321de44fd, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.012235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.090392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1134/2267] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:51 2022] PUT /identity/v3/projects/39115160ca364b2590af12fccde98256/users/78888cc145ba4052be05b1c321de44fd/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:52.093740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1134/2268] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:03:51 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:03:52.094999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.095147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.095453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.095785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.117523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:52.118881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:52.166786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8fd30-5278-4f43-9c7c-50467ca7c29b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.172960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.173651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.173861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.174147 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.174978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.176674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.190876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1135/2269] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:03:52.191484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1135/2270] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:52.191944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.198491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.199950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:52.202711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c643b-a374-4ab7-b34a-8dd43155dfe5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.207467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.208154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.208432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.208719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] PATH_INFO: `/v3/projects/362540e3e00340d99eec9eb966c70ad1/users/1efa854c69ec40b59c63d8a11fcb487e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.225920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] RBAC: Authorizing `identity:create_grant(project_id=362540e3e00340d99eec9eb966c70ad1, user_id=1efa854c69ec40b59c63d8a11fcb487e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.228716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.260941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5e3ed-0375-4954-b784-f8241cc28461 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.266432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.267058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.267345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.267648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.277948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.279574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.291214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1136/2271] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:52.300383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1136/2272] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:52 2022] PUT /identity/v3/projects/362540e3e00340d99eec9eb966c70ad1/users/1efa854c69ec40b59c63d8a11fcb487e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:03:52.304817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.314889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090941d2-8714-42b4-8a38-6b7d88111f78 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.376803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.377344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.377636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.377914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.378789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.380304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cbb16-0474-4952-8e1f-89492510b3f8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.389902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1137/2273] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:03:52.394190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.405716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6d025-99fe-4c12-9b5d-8564e00bdede admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.410606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.411683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.411683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.411971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] PATH_INFO: `/v3/projects/362540e3e00340d99eec9eb966c70ad1/users/1efa854c69ec40b59c63d8a11fcb487e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.419484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362540e3e00340d99eec9eb966c70ad1, user_id=1efa854c69ec40b59c63d8a11fcb487e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.419484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.428840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1137/2274] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:03:52 2022] PUT /identity/v3/projects/362540e3e00340d99eec9eb966c70ad1/users/1efa854c69ec40b59c63d8a11fcb487e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:52.439732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.440027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.440317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.475023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] MFA Rules not processed for user `1efa854c69ec40b59c63d8a11fcb487e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:52.476294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:52.510585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:52.550998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1138/2275] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:52.562494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.564299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:52.631683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dab4c-49e5-4d16-b42c-db83f9194f25 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.641358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.642083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.642389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.642675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:52.662049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:52.662049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:52.664843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1138/2276] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:52.860407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:52.947504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:52.949147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:52.962156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a255a4c-0b4f-4b60-9aad-36f1302cdc8d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:52.966479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:52.967101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:52.967387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:52.967683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:53.028459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:53.030059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6beddde-62fe-41c5-9950-e22f6e056469 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:53.039197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1139/2277] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:54.103757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:54.335538 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:56.110204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:56.591523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:57.201378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:58.088811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:58.233792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9081-3e31-4556-a54a-126ec2616007 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:58.239467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:58.240165 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:58.240481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:58.240738 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] PATH_INFO: `/v3/users/3ffc1548791e4762adea85a390764cf0` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:58.242295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ffc1548791e4762adea85a390764cf0)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:58.245607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:58.528886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ffc1548791e4762adea85a390764cf0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:58.537184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ffc1548791e4762adea85a390764cf0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:58.537184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ffc1548791e4762adea85a390764cf0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:58.537184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:58.560323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ffc1548791e4762adea85a390764cf0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:58.685178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ffc1548791e4762adea85a390764cf0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:58.702828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1139/2278] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:03:58 2022] DELETE /identity/v3/users/3ffc1548791e4762adea85a390764cf0 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:03:58.707463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:58.708787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:58.709224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:58.738906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:58.740839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:58.816199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1140/2279] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:58.827055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:58.828720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:58.936320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559101f-e23b-4b55-a339-9fc0c7af90a0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:58.941293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:58.942181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:58.942554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:58.942939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:58.955138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:58.956751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53f74f-9209-493b-8ddf-13c324953a18 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:58.969074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1140/2280] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:59.548946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:59.551392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:59.571330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cf732-e999-4bbd-a59c-13dd21e0985a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:59.590309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:59.591343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:59.591625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:59.591902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] PATH_INFO: `/v3/projects/9c7e51fb281446fdad1e82543789b6e9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:59.594080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c7e51fb281446fdad1e82543789b6e9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:59.598509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:59.792515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c7e51fb281446fdad1e82543789b6e9', 'request_id': 'req-85516d77-2240-4e9c-bc80-c4246d4179eb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:59.800371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c7e51fb281446fdad1e82543789b6e9', 'request_id': 'req-85516d77-2240-4e9c-bc80-c4246d4179eb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:59.827127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c7e51fb281446fdad1e82543789b6e9', 'request_id': 'req-85516d77-2240-4e9c-bc80-c4246d4179eb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:03:59.942912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:59.943394 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:59.944100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:59.975231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] MFA Rules not processed for user `78888cc145ba4052be05b1c321de44fd`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:59.976458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:00.011724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] The token cache is being invalidate because project 9c7e51fb281446fdad1e82543789b6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:00.011921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.017687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.025740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1141/2281] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:03:59 2022] DELETE /identity/v3/projects/9c7e51fb281446fdad1e82543789b6e9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:00.027065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:00.030405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:00.104986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:00.151376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1141/2282] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:00.154471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:00.156870 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:00.173640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85516d77-2240-4e9c-bc80-c4246d4179eb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:00.179111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:00.179732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.180227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.180685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] PATH_INFO: `/v3/users/1efa854c69ec40b59c63d8a11fcb487e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.183388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efa854c69ec40b59c63d8a11fcb487e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:00.185328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:00.202126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897519-a515-45cf-aee3-3ac15df050a0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:00.221366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:00.222111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.222499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.223014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.224021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:00.226400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:00.373116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1142/2283] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:00 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:00.378157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:00.451436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:00.480839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efa854c69ec40b59c63d8a11fcb487e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.481727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efa854c69ec40b59c63d8a11fcb487e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.529120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c29d28-37e0-45a4-9065-a0213f5fea60 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:00.534480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:00.535498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.535966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.536395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.537225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:00.540701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:00.579290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efa854c69ec40b59c63d8a11fcb487e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.607338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efa854c69ec40b59c63d8a11fcb487e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.608099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efa854c69ec40b59c63d8a11fcb487e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:00.632052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1142/2284] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:00 2022] DELETE /identity/v3/users/1efa854c69ec40b59c63d8a11fcb487e => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:00.645367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1143/2285] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:00 2022] POST /identity/v3/users => generated 298 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:00.664140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:00.668655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:00.780552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc53101-291d-4cc9-acc2-818d1628581f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:00.786513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:00.787122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.787409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.788141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.789839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:00.791059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:00.806605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1143/2286] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:00.808361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:00.820361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2719-ef75-40eb-bfc4-02280464471e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:00.826549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:00.827365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.827716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.828075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] PATH_INFO: `/v3/projects/54e11078158541c0bafc3b82d2da2d72/users/7367c39686814c91aa5e1fd8eff6eb06/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.864896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e11078158541c0bafc3b82d2da2d72, user_id=7367c39686814c91aa5e1fd8eff6eb06, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:00.864896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.028975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1144/2287] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:00 2022] PUT /identity/v3/projects/54e11078158541c0bafc3b82d2da2d72/users/7367c39686814c91aa5e1fd8eff6eb06/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:01.032944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.058066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663d50-6384-4756-8bca-e64809f90a0d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:01.077759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.083925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:01.173932 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:01.175287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.175762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.188067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.188067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:01.188067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.203299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1144/2288] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:01.208318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.228917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b0501-2bc4-416c-90e7-254752af2041 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:01.244355 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49586-9c43-4afb-b6d4-cd242acbbe28 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:01.245319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:01.246206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.246520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.246805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] PATH_INFO: `/v3/projects/54e11078158541c0bafc3b82d2da2d72/users/7367c39686814c91aa5e1fd8eff6eb06/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.256268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e11078158541c0bafc3b82d2da2d72, user_id=7367c39686814c91aa5e1fd8eff6eb06, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:01.256268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.260900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:01.261917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.262237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.262398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] PATH_INFO: `/v3/users/c91083376132452693a47d709fe85ef7` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.264678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91083376132452693a47d709fe85ef7)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:01.266873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.291556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1145/2289] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:01 2022] PUT /identity/v3/projects/54e11078158541c0bafc3b82d2da2d72/users/7367c39686814c91aa5e1fd8eff6eb06/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:01.534507 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91083376132452693a47d709fe85ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.535370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91083376132452693a47d709fe85ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.538787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91083376132452693a47d709fe85ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.558260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.559588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:01.572478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91083376132452693a47d709fe85ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.660964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-73c02894-9c91-42b3-a242-1f17c21cfe77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91083376132452693a47d709fe85ef7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.690635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1145/2290] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:01 2022] DELETE /identity/v3/users/c91083376132452693a47d709fe85ef7 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:01.693358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.693358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.693358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.699295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0693-24de-49fb-9709-1a65e2c95e3e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:01.742214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:01.763005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:01.804092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:01.805804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.806873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.807802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] PATH_INFO: `/v3/projects/362540e3e00340d99eec9eb966c70ad1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.810639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=362540e3e00340d99eec9eb966c70ad1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:01.813040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.902469 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:01.942523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1146/2291] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:01.980129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362540e3e00340d99eec9eb966c70ad1', 'request_id': 'req-7cefb0cb-f647-47bb-bfa6-69829188ee15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:01.984985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.986493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:02.013846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362540e3e00340d99eec9eb966c70ad1', 'request_id': 'req-7cefb0cb-f647-47bb-bfa6-69829188ee15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:02.025415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362540e3e00340d99eec9eb966c70ad1', 'request_id': 'req-7cefb0cb-f647-47bb-bfa6-69829188ee15'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:02.110437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] The token cache is being invalidate because project 362540e3e00340d99eec9eb966c70ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:02.110931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:02.112193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7cefb0cb-f647-47bb-bfa6-69829188ee15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:02.115042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1146/2292] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:01 2022] DELETE /identity/v3/projects/362540e3e00340d99eec9eb966c70ad1 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:02.120730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db218-6a5a-4734-a6c1-f61307021850 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.121231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db218-6a5a-4734-a6c1-f61307021850 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.121664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db218-6a5a-4734-a6c1-f61307021850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.129114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e299c-0681-4805-8eec-f39f0626ef32 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.152690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-591db218-6a5a-4734-a6c1-f61307021850 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:02.155046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-591db218-6a5a-4734-a6c1-f61307021850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:02.255735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.256554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.256908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.257025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1147/2293] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:02.257303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.264235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:02.265655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.265958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.266264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.287323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:02.288930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:02.294520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.298920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.307309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1148/2294] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:02.307309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.317191 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1147/2295] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:02.323340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5341-978c-4798-8435-4f219ddc3dab None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.339545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.340476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.341750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.343023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.343974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:02.345300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.351432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.380900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1149/2296] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:02.391345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.406618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4980ba-ffcf-4787-ac22-f22182f18e04 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.423986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.424960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.425454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.427233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.428265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.430533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.440445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:02.573271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1148/2297] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:02 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:02.577862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.590634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18615-854a-40db-8468-d4817b6064aa admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.700946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.701743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.702447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.702447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.703113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.705109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.843816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1150/2298] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:02 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:02.845971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.859803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf631-230b-4ab7-927d-b535f8959bc3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.865966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.867247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.868528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.869146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.871054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.874769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.905047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1149/2299] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:02.905393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.916949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfc9af-f3da-40b4-b943-81f5cb221722 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:02.928387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:02.929390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:02.934813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:02.935204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:02.936054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:02.950900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:02.956370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.959944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:02.969715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1151/2300] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:02.973756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:02.997397 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab2916-e102-4da6-80d0-8b28976b3931 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.012943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78463f-cf17-46e9-a8aa-f04c39f65da8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.025054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.025369 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.025670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.025951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] PATH_INFO: `/v3/projects/87094221cc514814ae30c6ebde90d8a9/users/cde8ccbf0e8242ad9e5d60114db42768/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.027108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.030002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.031397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.031759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] PATH_INFO: `/v3/projects/5df2cf30f5194eb58881a670ba990acd` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.033614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df2cf30f5194eb58881a670ba990acd)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.040471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.067972 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87094221cc514814ae30c6ebde90d8a9, user_id=cde8ccbf0e8242ad9e5d60114db42768, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.077394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fcc36-3b60-4f6c-84fc-5b73c7af204e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.148758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1152/2301] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:02 2022] PUT /identity/v3/projects/87094221cc514814ae30c6ebde90d8a9/users/cde8ccbf0e8242ad9e5d60114db42768/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:03.160699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.160986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.161431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.196641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] MFA Rules not processed for user `cde8ccbf0e8242ad9e5d60114db42768`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:03.199561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:03.210589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df2cf30f5194eb58881a670ba990acd', 'request_id': 'req-a7bd82f1-59ae-4a62-9f66-31b743049825'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.224976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df2cf30f5194eb58881a670ba990acd', 'request_id': 'req-a7bd82f1-59ae-4a62-9f66-31b743049825'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.300501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df2cf30f5194eb58881a670ba990acd', 'request_id': 'req-a7bd82f1-59ae-4a62-9f66-31b743049825'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.317162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] The token cache is being invalidate because project 5df2cf30f5194eb58881a670ba990acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:03.317718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.318235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.322699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1150/2302] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:02 2022] DELETE /identity/v3/projects/5df2cf30f5194eb58881a670ba990acd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:03.322879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1153/2303] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:03.322993 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.324086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.325047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:03.431188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd82f1-59ae-4a62-9f66-31b743049825 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.460386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.461324 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.461799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.462429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.467163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.467163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.474019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4198bf-3994-4610-bb74-ee6f4f183d70 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.485491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.486637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.486637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.486862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] PATH_INFO: `/v3/users/81397ee6bd284cceba99679b57c4a843` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.487994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81397ee6bd284cceba99679b57c4a843)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.490124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.623598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1151/2304] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:03 2022] POST /identity/v3/projects => generated 369 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:03.627763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.722448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a15cdb-d1f0-4494-8f98-b2c5a83f2938 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.728396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.728713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.728937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.729221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.729911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.732056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.776807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81397ee6bd284cceba99679b57c4a843'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.777716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81397ee6bd284cceba99679b57c4a843'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.829514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1152/2305] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:03 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:03.833154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.848466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81397ee6bd284cceba99679b57c4a843'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.861890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befaf7e-aaed-4118-b64e-c2f0949bfa84 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.869001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81397ee6bd284cceba99679b57c4a843'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.870134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81397ee6bd284cceba99679b57c4a843'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:03.872571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.873535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.874081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.874684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.875684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.877479 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.886233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1154/2306] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:03 2022] DELETE /identity/v3/users/81397ee6bd284cceba99679b57c4a843 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:03.906759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1153/2307] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:04:03.911855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.915068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:04.009538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3009534-c4d3-4c3d-9437-aaf52ea3f377 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:04.014563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:04.015239 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.015520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.015796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] PATH_INFO: `/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8/users/beffec21973d42598c0a27480f7a9465/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.038458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ece67d9c5b445c1bfee415af7fbfbf8, user_id=beffec21973d42598c0a27480f7a9465, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:04.038458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:04.294243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1155/2308] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:03 2022] PUT /identity/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8/users/beffec21973d42598c0a27480f7a9465/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:04.294734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:04.305723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c8c72-9474-43a7-9e05-ce9cdcca4f17 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:04.394137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:04.394825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.395438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.395438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.396477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:04.399104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c52e52-8c2d-4caf-a852-8b344d00e55b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:04.411326 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1154/2309] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:04.416347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:04.427378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230d21-d12d-47dd-848d-fe8e226542b2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:04.434139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:04.435028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.435315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.435596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] PATH_INFO: `/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8/users/beffec21973d42598c0a27480f7a9465/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.438479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ece67d9c5b445c1bfee415af7fbfbf8, user_id=beffec21973d42598c0a27480f7a9465, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:04.441214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:04.465856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1156/2310] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:04 2022] PUT /identity/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8/users/beffec21973d42598c0a27480f7a9465/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:04.477075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.477412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.477706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.510953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] MFA Rules not processed for user `beffec21973d42598c0a27480f7a9465`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:04.512128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:04.591663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1155/2311] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:04.604038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:04.611070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:04.615657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:04.622117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:04.753742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c506fc-bfaf-4445-97d6-dbd3b819887f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:04.779790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:04.779790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.780088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.780088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.799405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:04.819007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:04.820913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a32f19-d4c2-48e0-80e4-56e0ed37f091 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:04.827150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abdb04-2f4b-4e18-99f8-0e6cd32737e0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:04.859396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1157/2312] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:04.873371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:04.874254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:04.874335 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:04.874655 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] PATH_INFO: `/v3/projects/e5a859a37406451fb33f2c716a944ffb` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:04.884207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a859a37406451fb33f2c716a944ffb)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:04.891604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:05.069439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a859a37406451fb33f2c716a944ffb', 'request_id': 'req-7f80210a-c53c-4243-a0da-abe4edf3eae2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:05.078120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a859a37406451fb33f2c716a944ffb', 'request_id': 'req-7f80210a-c53c-4243-a0da-abe4edf3eae2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:05.173939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a859a37406451fb33f2c716a944ffb', 'request_id': 'req-7f80210a-c53c-4243-a0da-abe4edf3eae2'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:05.187894 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] The token cache is being invalidate because project e5a859a37406451fb33f2c716a944ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:05.188208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:05.188878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7f80210a-c53c-4243-a0da-abe4edf3eae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:05.190833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1156/2313] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:04 2022] DELETE /identity/v3/projects/e5a859a37406451fb33f2c716a944ffb => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:05.196088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.196399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.196676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.215779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:05.217046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:05.354554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1158/2314] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:05.363094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.364056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.364338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.388865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:05.390300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:05.402490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1157/2315] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:05.404491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:05.415559 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ef7bd-aba2-4140-b743-9a5f8c7a0bde None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:05.420453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:05.421088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.421377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.421661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.422410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:05.422817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:05.424360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:05.438879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1159/2316] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:05.443540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:05.458761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1259501-48f5-47fa-9d23-84cbee710e1a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:05.466295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:05.467777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.467777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.467777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.468237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:05.470166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:05.601987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1158/2317] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:05 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:05.605421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:05.622385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f410ea-be97-49c2-b734-e9385506be4e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:05.710448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:05.711714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.712003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.712282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.713231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:05.716091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:05.756931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:05.884291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1160/2318] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:05 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:05.885171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:05.909859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f0ea-ffc6-495a-ad51-205f93532546 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:05.911927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:05.912736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.913124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.913504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.915125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:05.917100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:05.963859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1159/2319] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:04:05.974772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:05.995299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b464-b7b0-4b4b-8d10-105ff256265e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.011377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.012143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.012901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.013291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.014286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.028876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d02813-7c66-4a9a-8a53-cc2324dc6e34 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.037057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1161/2320] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:06.043689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.062428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070ada-6738-4bc1-98a8-cec1038cfc51 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.074008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.076430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.076430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.076430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] PATH_INFO: `/v3/projects/5f29656fa3a44cd8a2707a197a56dcd8/users/293c3895eb9240f5aff70036301a5921/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.109034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f29656fa3a44cd8a2707a197a56dcd8, user_id=293c3895eb9240f5aff70036301a5921, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.112748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.234371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1160/2321] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:06 2022] PUT /identity/v3/projects/5f29656fa3a44cd8a2707a197a56dcd8/users/293c3895eb9240f5aff70036301a5921/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:06.248070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.248365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.248641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.299975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] MFA Rules not processed for user `293c3895eb9240f5aff70036301a5921`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:06.299975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:06.470219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1162/2322] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:06.474838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.583210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9470c7e-6fc2-4b53-87ab-3dda18bd8bf0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.594200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.595251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.595811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.596365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.598539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.601286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.730389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1161/2323] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:06 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:06.734417 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.747545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068b644-b59a-4f4a-88ef-d52a7220e6de None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.854188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.856961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.857756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.858056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.860713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.860713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.861567 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.862913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:06.894400 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975991a-df00-42c2-9490-f2fede96bf79 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.912493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.912493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.912493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.912493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] PATH_INFO: `/v3/users/16e6308e10cd46eea31c48a02c0a9e37` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.913157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e6308e10cd46eea31c48a02c0a9e37)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.916221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.967415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1163/2324] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:06 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:06.970973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.983006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e263ae-efd9-4212-8dc4-60f5a4a54010 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.993892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.993892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.993892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.993892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.994257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.994257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.006390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1164/2325] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:04:07.010461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.025171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a5a7e-70f0-428e-9a6d-1b9cba25029b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:07.033640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:07.034689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.034980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.035394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.035964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:07.038252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.056599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1165/2326] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:07.070829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.084209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790cc1d-3564-4592-8fd2-e113e415e58d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:07.090814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:07.092044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.092531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.093011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] PATH_INFO: `/v3/projects/d55b6aef101947caa0160045d81b1193/users/c87a083642834da6829614f3ca028e3b/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.110843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55b6aef101947caa0160045d81b1193, user_id=c87a083642834da6829614f3ca028e3b, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:07.114665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef323d9b-fc06-4daa-abca-1d9138c3920a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.176867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e6308e10cd46eea31c48a02c0a9e37'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:07.177943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e6308e10cd46eea31c48a02c0a9e37'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:07.178842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e6308e10cd46eea31c48a02c0a9e37'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:07.229154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1166/2327] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:07 2022] PUT /identity/v3/projects/d55b6aef101947caa0160045d81b1193/users/c87a083642834da6829614f3ca028e3b/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:07.231239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e6308e10cd46eea31c48a02c0a9e37'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:07.240481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.240853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.241215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.274842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] MFA Rules not processed for user `c87a083642834da6829614f3ca028e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:07.280235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:07.337739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74b9e470-b3d6-4ea5-8bda-97f5f92dc7d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e6308e10cd46eea31c48a02c0a9e37'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:07.363990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1162/2328] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:06 2022] DELETE /identity/v3/users/16e6308e10cd46eea31c48a02c0a9e37 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:07.370681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.370979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.371267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.390100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1167/2329] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:07.398993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.401859 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:07.403033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:07.403292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:07.499377 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1163/2330] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:07.501740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338018b-90ed-4046-8b8f-b9b1b96e4757 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:07.505148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.510459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:07.515875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:07.516547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.518400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.518400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.520803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:07.524625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.616668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b19d6-8cb1-447f-bdaf-0cb260395c24 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:07.623824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:07.624761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.625225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.625674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.642648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:07.644496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.648682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1168/2331] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:07 2022] POST /identity/v3/projects => generated 377 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:07.652739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.687271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1164/2332] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:07.750849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5221ee-7713-40f0-93f6-4e5c645f3b63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:07.760265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:07.761624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:07.762130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:07.762819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:07.764073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:07.767250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:07.972011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1169/2333] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:07 2022] POST /identity/v3/users => generated 295 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:07.977454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:07.995162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b903768-7c16-4d1f-b13c-c39c4066aad4 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:08.005761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:08.007372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.009306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.014048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.015722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:08.053305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:08.098824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:08.111812 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1165/2334] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:08.116189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cf548-7e23-4564-90d3-1114d84c34f0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:08.136632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:08.140303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.140303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.140303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] PATH_INFO: `/v3/projects/3250a40b6c3848309dad07640a66df93/users/a09e34d183c34dd6bfe2733e9abad20a/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.175527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3250a40b6c3848309dad07640a66df93, user_id=a09e34d183c34dd6bfe2733e9abad20a, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:08.178435 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:08.287259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1170/2335] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:08 2022] PUT /identity/v3/projects/3250a40b6c3848309dad07640a66df93/users/a09e34d183c34dd6bfe2733e9abad20a/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:08.289636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:08.317790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b52e3-fb74-49e6-8cca-2c1d6b2639e5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:08.360438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:08.438211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:08.439262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.439448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.439740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.441086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:08.443831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3505d6-f0bd-4fa6-8bdf-6c10c6a8f867 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:08.460113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1166/2336] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:08.465916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:08.480257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb6d52-1922-409e-8d04-16f44a92f85e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:08.486018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:08.495661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.495661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.495661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] PATH_INFO: `/v3/projects/3250a40b6c3848309dad07640a66df93/users/a09e34d183c34dd6bfe2733e9abad20a/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.495661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3250a40b6c3848309dad07640a66df93, user_id=a09e34d183c34dd6bfe2733e9abad20a, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:08.495661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:08.521293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1171/2337] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:08 2022] PUT /identity/v3/projects/3250a40b6c3848309dad07640a66df93/users/a09e34d183c34dd6bfe2733e9abad20a/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:08.532161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.532689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.532796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.605788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] MFA Rules not processed for user `a09e34d183c34dd6bfe2733e9abad20a`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:08.608429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:08.717629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1167/2338] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:08.744426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:08.747302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:08.961116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d153-c043-4005-8cf3-c7fd7ed3a1ef admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:08.966039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:08.967199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:08.973912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:08.973912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:08.975070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:08.999503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:09.041780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:09.043884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:09.103147 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:09.105191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:09.115483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1172/2339] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:09.164811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa1438-4856-4bd2-9514-56119ce187be None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:09.170979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:09.172070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:09.172193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:09.175782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] PATH_INFO: `/v3/projects/cd50183558114776b1db5fd3ee4ee7cf` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:09.175782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd50183558114776b1db5fd3ee4ee7cf)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:09.176206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:09.353908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd50183558114776b1db5fd3ee4ee7cf', 'request_id': 'req-83c1577c-0e22-412d-8aae-a971acb514f3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.361796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd50183558114776b1db5fd3ee4ee7cf', 'request_id': 'req-83c1577c-0e22-412d-8aae-a971acb514f3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.447512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd50183558114776b1db5fd3ee4ee7cf', 'request_id': 'req-83c1577c-0e22-412d-8aae-a971acb514f3'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.472205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] The token cache is being invalidate because project cd50183558114776b1db5fd3ee4ee7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:09.472599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.473806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.476807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1168/2340] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:09 2022] DELETE /identity/v3/projects/cd50183558114776b1db5fd3ee4ee7cf => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:09.481088 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:09.494060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cddcaf-50a0-4a93-96a3-1b3efffe9597 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:09.558976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:09.560593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:09.560593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:09.560593 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] PATH_INFO: `/v3/users/9f80b67312e048e195a20df202a28b8a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:09.561671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f80b67312e048e195a20df202a28b8a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:09.563801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:09.775796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f80b67312e048e195a20df202a28b8a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.776480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f80b67312e048e195a20df202a28b8a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.906477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f80b67312e048e195a20df202a28b8a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.944463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f80b67312e048e195a20df202a28b8a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.946363 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f80b67312e048e195a20df202a28b8a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:09.966536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1173/2341] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:09 2022] DELETE /identity/v3/users/9f80b67312e048e195a20df202a28b8a => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:10.094564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:10.228808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:10.230284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:10.263046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1577c-0e22-412d-8aae-a971acb514f3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:10.466690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:10.467365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:10.467648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:10.470725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] PATH_INFO: `/v3/users/cde8ccbf0e8242ad9e5d60114db42768` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:10.479868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde8ccbf0e8242ad9e5d60114db42768)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:10.482107 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:10.745989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:10.747831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:10.776915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1ee4f-100b-4792-abe5-10aecedce5d8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:10.786973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:10.789393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:10.790264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:10.790904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] PATH_INFO: `/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:10.793168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d0a093fe0949ffae320b1df0b4b2cd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:10.796394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:10.868797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde8ccbf0e8242ad9e5d60114db42768'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:10.870133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde8ccbf0e8242ad9e5d60114db42768'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:10.871865 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde8ccbf0e8242ad9e5d60114db42768'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:10.898222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde8ccbf0e8242ad9e5d60114db42768'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.022117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53a0f4a8-256f-4516-a1bf-e1f4d59c7d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde8ccbf0e8242ad9e5d60114db42768'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.039255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1169/2342] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:10 2022] DELETE /identity/v3/users/cde8ccbf0e8242ad9e5d60114db42768 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:11.051662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245475d-b025-45c7-bd5e-bf495cdd6804 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.051976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245475d-b025-45c7-bd5e-bf495cdd6804 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.052262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245475d-b025-45c7-bd5e-bf495cdd6804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.080421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d0a093fe0949ffae320b1df0b4b2cd', 'request_id': 'req-d02e0fde-08bb-4541-9170-73a46227d8e6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.087437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d0a093fe0949ffae320b1df0b4b2cd', 'request_id': 'req-d02e0fde-08bb-4541-9170-73a46227d8e6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.092742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5245475d-b025-45c7-bd5e-bf495cdd6804 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:11.093832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d0a093fe0949ffae320b1df0b4b2cd', 'request_id': 'req-d02e0fde-08bb-4541-9170-73a46227d8e6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.096158 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5245475d-b025-45c7-bd5e-bf495cdd6804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.173134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] The token cache is being invalidate because project a8d0a093fe0949ffae320b1df0b4b2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:11.174654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.176784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1170/2343] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:11.178228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.178563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.178846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.183333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:11.185032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1174/2344] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:10 2022] DELETE /identity/v3/projects/a8d0a093fe0949ffae320b1df0b4b2cd => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:11.186020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.188818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.224846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] MFA Rules not processed for user `7367c39686814c91aa5e1fd8eff6eb06`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:11.228964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.257849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e0fde-08bb-4541-9170-73a46227d8e6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.262093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.262752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.263079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.263527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.265387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.280379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.281866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d0b86-abd6-432b-a388-c05704a49961 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.295338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1175/2345] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:11.310933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097a38-5027-44eb-8f8b-8b6e0164846c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.311330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097a38-5027-44eb-8f8b-8b6e0164846c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.311675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097a38-5027-44eb-8f8b-8b6e0164846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.347915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bc097a38-5027-44eb-8f8b-8b6e0164846c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:11.349153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bc097a38-5027-44eb-8f8b-8b6e0164846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.371389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1176/2346] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:11.373339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1171/2347] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:11.380184 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.382536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.383168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.389428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.391653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.416508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:11.417729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.454873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1177/2348] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:11.456829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.470578 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a5ee2-3152-4a97-91bf-bcaf7f5d8048 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.481236 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.481861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.482208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.482618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.483751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:11.483991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.491494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.495543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181aee8f-b101-45f4-a044-0888ebb40293 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.500752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.501568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.501890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.502239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.504309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.507046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1178/2349] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:11.508694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.534167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad357e-2ee0-484f-b5db-f726f1ace7f4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.551576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.552228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.552516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.552822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.557960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.560040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.615799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.619062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.630963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1172/2350] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:11.631930 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.633954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.706282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669cecd-2d5a-4c4b-b4a3-02146e452f0b service glance] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.714067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.717405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.717405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.717405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.717984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.719023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1179/2351] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:11.723375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.827207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9885-6d0e-4003-8bdc-39c812071b3d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.847546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.848677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.848966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.849247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.849924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.852178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.854417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.856188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.870848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1173/2352] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:11.910072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.917512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.935154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:11.952049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65998b-1237-42ee-b3b7-726e60e9b816 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:11.965362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1180/2353] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:11.966043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:11.969430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.969430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.969430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] PATH_INFO: `/v3/projects/87094221cc514814ae30c6ebde90d8a9` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.969430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87094221cc514814ae30c6ebde90d8a9)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:11.971000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:11.975577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.009257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70b3b8-68db-4e17-9087-ed31c752b011 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.024862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.025542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.025823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.026312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.030806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.032441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dd984-0028-4d18-b8b9-4b6af3ab17c8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.046402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1181/2354] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:12.046402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.046402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.046402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.069494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:12.071592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.154992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1182/2355] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:04:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:12.161076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.167167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87094221cc514814ae30c6ebde90d8a9', 'request_id': 'req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.176786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90b8b8-d931-4bdc-8968-0ddadf0c3185 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.190248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87094221cc514814ae30c6ebde90d8a9', 'request_id': 'req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.262303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.263736 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.264026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.264313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.265027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.266690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.277260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1183/2356] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:12.278227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.280574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.281970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87094221cc514814ae30c6ebde90d8a9', 'request_id': 'req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.296339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] The token cache is being invalidate because project 87094221cc514814ae30c6ebde90d8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:12.296557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.297439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.299501 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1174/2357] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:11 2022] DELETE /identity/v3/projects/87094221cc514814ae30c6ebde90d8a9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:12.300577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.302100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.322939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67faa5a-e2b7-448f-ab07-edac26d5e3d7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.333256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.334265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.334743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.335227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] PATH_INFO: `/v3/projects/07776ffe18f540aaac15b1e39c0bf0b4/users/65437895c89743c1bf5980166acd9379/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.358646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07776ffe18f540aaac15b1e39c0bf0b4, user_id=65437895c89743c1bf5980166acd9379, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.362413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d56681-1b8f-4d5a-93f3-eef0984c403f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.378568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d1f21-ea51-47af-bc24-d8adfcdf469f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.383982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.384603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.384889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.385175 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.387025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.449687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.451566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.462595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1184/2358] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:04:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:04:12.470520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.470520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.472080 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1175/2359] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:12 2022] PUT /identity/v3/projects/07776ffe18f540aaac15b1e39c0bf0b4/users/65437895c89743c1bf5980166acd9379/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:12.474236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.474752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.475192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.499214 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19ac4-ada4-46d1-98be-f70399f58791 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.533968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] MFA Rules not processed for user `65437895c89743c1bf5980166acd9379`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:12.535597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.583500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.584597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.584892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.585218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] PATH_INFO: `/v3/users/beffec21973d42598c0a27480f7a9465` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.586585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=beffec21973d42598c0a27480f7a9465)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.588708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.641342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1176/2360] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:12.646562 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.668504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218cb3a-ab4f-4119-b7be-4cffbf406334 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.673612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.674297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.674611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.674895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.676189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.678328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.803465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1177/2361] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:12 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:12.808286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:12.885678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beffec21973d42598c0a27480f7a9465'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.886530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beffec21973d42598c0a27480f7a9465'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:12.968551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beffec21973d42598c0a27480f7a9465'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:13.046032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beffec21973d42598c0a27480f7a9465'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:13.051309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beffec21973d42598c0a27480f7a9465'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:13.078444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0239e-077b-45f0-b968-9c8048541f2e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:13.078444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:13.078444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:13.078444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:13.079654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:13.079654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:13.079654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:13.109367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1185/2362] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:12 2022] DELETE /identity/v3/users/beffec21973d42598c0a27480f7a9465 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:13.140032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:13.211539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1178/2363] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:12 2022] POST /identity/v3/users => generated 297 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:13.216112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:13.222855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:13.367427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37621a-ed03-48e6-b50a-f300eee779bd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:13.373910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:13.375755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:13.376221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:13.376678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:13.379079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:13.380187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:13.409632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:13.411421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1186/2364] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:13.432888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf83aa-66c5-481b-86f7-d644a74095a1 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:13.439551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:13.440329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:13.441162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:13.441162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] PATH_INFO: `/v3/projects/db0ab3e4124d402a8625e0648bc75857/users/82f8f519d2114338bbf7a56147e4ef2e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:13.479060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0ab3e4124d402a8625e0648bc75857, user_id=82f8f519d2114338bbf7a56147e4ef2e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:13.482905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:13.613725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1179/2365] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:13 2022] PUT /identity/v3/projects/db0ab3e4124d402a8625e0648bc75857/users/82f8f519d2114338bbf7a56147e4ef2e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:13.615656 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:13.659563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e12ed-c64c-4d2a-8837-a7474622e774 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:13.803285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:13.873444 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:13.874252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:13.874670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:13.875355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:13.876302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:13.879228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:13.915208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:13.916150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1187/2366] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:13.929444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d39c5-bdfe-40f1-b855-8a7e917b2288 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:13.952884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:13.953780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:13.954267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:13.954421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] PATH_INFO: `/v3/projects/db0ab3e4124d402a8625e0648bc75857/users/82f8f519d2114338bbf7a56147e4ef2e/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:13.956788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0ab3e4124d402a8625e0648bc75857, user_id=82f8f519d2114338bbf7a56147e4ef2e, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:13.959583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6169-7fcb-440c-ab01-6d09e743d27c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:13.980339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:13.982382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:13.989141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1180/2367] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:13 2022] PUT /identity/v3/projects/db0ab3e4124d402a8625e0648bc75857/users/82f8f519d2114338bbf7a56147e4ef2e/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:14.000389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.000686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.000970 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.010293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daedd11-b125-460b-b080-3373c32ab829 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:14.073787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] MFA Rules not processed for user `82f8f519d2114338bbf7a56147e4ef2e`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:14.076527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:14.164516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:14.165275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.167161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.167912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] PATH_INFO: `/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.170277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ece67d9c5b445c1bfee415af7fbfbf8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:14.175653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:14.222467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1181/2368] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:14.271043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:14.272459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:14.345164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ece67d9c5b445c1bfee415af7fbfbf8', 'request_id': 'req-7402d820-a474-4369-ab97-307fa39ab894'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:14.355406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ece67d9c5b445c1bfee415af7fbfbf8', 'request_id': 'req-7402d820-a474-4369-ab97-307fa39ab894'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:14.363258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ece67d9c5b445c1bfee415af7fbfbf8', 'request_id': 'req-7402d820-a474-4369-ab97-307fa39ab894'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:14.403543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65d9f3-38a9-4c90-a209-bb063a46c76f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:14.452658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] The token cache is being invalidate because project 5ece67d9c5b445c1bfee415af7fbfbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:14.453188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:14.453869 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7402d820-a474-4369-ab97-307fa39ab894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:14.457889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1188/2369] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:13 2022] DELETE /identity/v3/projects/5ece67d9c5b445c1bfee415af7fbfbf8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:14.463479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.464410 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.465895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.488531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:14.489833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.491302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.492292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.494622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:14.605937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:14.705491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:14.732773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:14.734912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5166b9-5af7-4cc6-b211-e6d679141c79 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:14.781343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1182/2370] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:14.842050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1189/2371] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:14.844518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.845559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.846284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.895017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:14.897192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:14.921647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1183/2372] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:14.926572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:14.961799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eaef8-90c5-4d5a-9f69-ed9f6831427c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:14.969115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752904-8101-4350-b939-414d92811354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752904-8101-4350-b939-414d92811354 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752904-8101-4350-b939-414d92811354 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c752904-8101-4350-b939-414d92811354 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752904-8101-4350-b939-414d92811354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752904-8101-4350-b939-414d92811354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:14.974979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c752904-8101-4350-b939-414d92811354 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:14.998545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:14.998870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1190/2373] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:15.017409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0925b7-733b-494e-ac7c-cd51f6648533 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.034274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.035544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.037817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.038312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.039438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.044585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:15.244878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1184/2374] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:14 2022] POST /identity/v3/projects => generated 383 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:15.247508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752904-8101-4350-b939-414d92811354 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.284056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c752904-8101-4350-b939-414d92811354 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.371273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:15.512858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.514739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.514739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.515089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.516740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.523929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:15.667957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1191/2375] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:15 2022] POST /identity/v3/users => generated 292 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:15.672203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.706223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a96015-b4da-4097-b074-ae247d16c5f0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.722397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.723792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.724329 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.724836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.725863 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.730640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:15.767878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1185/2376] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:15.771682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.798415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33858a1b-2877-40f3-8995-992ebdbbaa16 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.808863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.809528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.809821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.810163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.810958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.812613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:15.844416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1192/2377] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:15.855308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.871402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1dd1d-73ab-4500-91ec-a7f8866684bb admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.876915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.878944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:15.895803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.896915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.896915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.897263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] PATH_INFO: `/v3/projects/2d2056635d6a41f0a5a02c1dcb27baa8/users/195fc3305bc84c8f9afe2fda85130c9a/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.957292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2056635d6a41f0a5a02c1dcb27baa8, user_id=195fc3305bc84c8f9afe2fda85130c9a, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.962478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f13bb-8939-4fe3-b7b2-eae1293e2645 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.054451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1186/2378] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:15 2022] PUT /identity/v3/projects/2d2056635d6a41f0a5a02c1dcb27baa8/users/195fc3305bc84c8f9afe2fda85130c9a/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:16.057653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264bd4f-73f5-47c0-8817-6b42f6ef0e4d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.071644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.079333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.079333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.143491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] MFA Rules not processed for user `195fc3305bc84c8f9afe2fda85130c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:16.146523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:16.169896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.170586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.170876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.171188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.176273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:16.311139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.319025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.324619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1187/2379] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:16.326146 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.341686 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1193/2380] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:16.465595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb45ec2-f6ef-4f8d-950f-c474894fdad5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.473065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.473981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.474459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.474902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.475736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.483836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.588771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.615463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1188/2381] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:16 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:16.619227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.704439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f63f14-7369-47f2-b7e1-abef4abf61cc service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.731798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a5f0-34d8-4d92-9cab-b869b017cdcb admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.739288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.740718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.742044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.742691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.744518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.747540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.800748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82f8f519d2114338bbf7a56147e4ef2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db0ab3e4124d402a8625e0648bc75857', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.801496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.801893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.803666 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.809709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:16.811225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.815457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.830959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1194/2382] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:16.836249 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.845325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1189/2383] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:16 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:04:16.849868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.879662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4398d-bc66-4d03-893c-b75725b7537c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.890551 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecafe8-f10e-4e9c-9f9e-eed6c1db1bab tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.899066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.899781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.900192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.900599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.901368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.904331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.916860 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82f8f519d2114338bbf7a56147e4ef2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db0ab3e4124d402a8625e0648bc75857', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.921053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.921053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.921053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.921053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.921394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:16.931831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1190/2384] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:16.956946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:16.975314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced5c3-be41-4c05-8718-b7b6961422a5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:16.984380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:16.985406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:16.985931 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:16.986420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:16.987518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:16.990851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.011649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1191/2385] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:17.015417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.029741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0dbf5-0c30-4e13-9021-bf25ea0f71a2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:17.035091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:17.035893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.036192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.036546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] PATH_INFO: `/v3/projects/09c20714f0d44e788c7e12b31b7411e7/users/95bf50a5409b4e9da6654913db2dfdbd/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.059337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c20714f0d44e788c7e12b31b7411e7, user_id=95bf50a5409b4e9da6654913db2dfdbd, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:17.062532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f00b78-1f1a-42fe-8db4-7d0c889674f5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.067942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1195/2386] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:16 2022] POST /identity/v3/projects => generated 353 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:17.072639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.196853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1192/2387] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:17 2022] PUT /identity/v3/projects/09c20714f0d44e788c7e12b31b7411e7/users/95bf50a5409b4e9da6654913db2dfdbd/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:17.204588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5d6ea-234a-4ac3-a8de-51e2ab258da1 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:17.207386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.207925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.208322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.270262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] MFA Rules not processed for user `95bf50a5409b4e9da6654913db2dfdbd`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:17.272997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:17.306758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:17.368183 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82f8f519d2114338bbf7a56147e4ef2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db0ab3e4124d402a8625e0648bc75857', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:17.369090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.369557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.370097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.371000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:17.371481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:17.377527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.394061 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:17.401257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1193/2388] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:17.404574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.405491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1196/2389] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:17.478146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.503575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0846b6-2084-4f20-a18b-850af014f348 tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:17.521986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82f8f519d2114338bbf7a56147e4ef2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db0ab3e4124d402a8625e0648bc75857', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:17.522872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.523237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.523642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] PATH_INFO: `/v3/projects/d100a04dbf2c4821a5ca28f7d0170f95` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.526268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d100a04dbf2c4821a5ca28f7d0170f95)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:17.531500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.586438 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266321-26b6-4578-a96d-4ecd12370462 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:17.593477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:17.594167 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.594492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.594780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.600318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:17.600318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.703702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd100a04dbf2c4821a5ca28f7d0170f95', 'request_id': 'req-b61779ea-4487-4f1b-880f-88c76e983bce'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:17.710007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1194/2390] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:17 2022] POST /identity/v3/projects => generated 387 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:17.713437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd100a04dbf2c4821a5ca28f7d0170f95', 'request_id': 'req-b61779ea-4487-4f1b-880f-88c76e983bce'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:17.715027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.721644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd100a04dbf2c4821a5ca28f7d0170f95', 'request_id': 'req-b61779ea-4487-4f1b-880f-88c76e983bce'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:17.803044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d3d2-897e-4b8c-9312-21adee868ce3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:17.809064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:17.809720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:17.810015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:17.810297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:17.811027 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:17.813273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:17.856614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] The token cache is being invalidate because project d100a04dbf2c4821a5ca28f7d0170f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:17.856930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:17.858652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:17.860426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1197/2391] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:17 2022] DELETE /identity/v3/projects/d100a04dbf2c4821a5ca28f7d0170f95 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:17.942762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1195/2392] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:17 2022] POST /identity/v3/users => generated 300 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:17.947228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:17.950086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:18.050826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61779ea-4487-4f1b-880f-88c76e983bce tempest-VolumeQuotaClassesTest-1858200129 tempest-VolumeQuotaClassesTest-1858200129-project-admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:18.063027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:18.063873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.064227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.064595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.065390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:18.067233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:18.080717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1198/2393] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:18.085345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:18.099767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33855-8858-4dac-9dc2-9b72605d74d0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:18.111064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:18.111064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.111456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.111696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] PATH_INFO: `/v3/projects/37d87044285a400eab159d905d21b58a/users/2ece22462b4a46579c2559c8e06fd394/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.144853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d87044285a400eab159d905d21b58a, user_id=2ece22462b4a46579c2559c8e06fd394, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:18.147717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:18.240439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1196/2394] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:18 2022] PUT /identity/v3/projects/37d87044285a400eab159d905d21b58a/users/2ece22462b4a46579c2559c8e06fd394/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:18.248549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:18.266873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08acb34-eaaf-483c-a8cf-fbf178ccf8fe admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:18.342599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:18.343241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.365161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.365161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.365161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:18.365161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042d58b-d877-4bab-adcb-5ac880b01143 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:18.365161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1199/2395] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:18.369214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:18.380525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f3a8-ff0a-41ee-a6da-28ead90002c2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:18.386208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:18.387041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.387390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.387718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] PATH_INFO: `/v3/projects/37d87044285a400eab159d905d21b58a/users/2ece22462b4a46579c2559c8e06fd394/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.389999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d87044285a400eab159d905d21b58a, user_id=2ece22462b4a46579c2559c8e06fd394, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:18.392761 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:18.404187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1197/2396] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:18 2022] PUT /identity/v3/projects/37d87044285a400eab159d905d21b58a/users/2ece22462b4a46579c2559c8e06fd394/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:18.416247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529bc7-fd05-462a-8526-e2befea22aa9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.416547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529bc7-fd05-462a-8526-e2befea22aa9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.416880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0529bc7-fd05-462a-8526-e2befea22aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.445114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f0529bc7-fd05-462a-8526-e2befea22aa9 None None] MFA Rules not processed for user `2ece22462b4a46579c2559c8e06fd394`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:18.446423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f0529bc7-fd05-462a-8526-e2befea22aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:18.520641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1200/2397] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:18.842197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:18.844539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:18.951533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e1c9e-fb9d-4545-bd7e-fc8b7338ce7d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:18.957282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:18.957910 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:18.958086 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:18.958396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:18.973528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:18.975076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:18.987490 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1198/2398] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:19.256598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:19.270640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722a7a-f151-4ee4-a1e6-cf681c26f551 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:19.270930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722a7a-f151-4ee4-a1e6-cf681c26f551 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:19.271212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722a7a-f151-4ee4-a1e6-cf681c26f551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:19.293621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53722a7a-f151-4ee4-a1e6-cf681c26f551 None None] MFA Rules not processed for user `564e0f9000a1448ea9ea37a2892e6495`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:19.294834 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53722a7a-f151-4ee4-a1e6-cf681c26f551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:19.361021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1201/2399] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:19.889044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:19.890555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:19.941804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:20.009002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac82b9-6b06-498c-a8e4-b0310611e19b service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:20.015103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:20.016510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:20.016883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:20.017477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] PATH_INFO: `/v3/users/65437895c89743c1bf5980166acd9379` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:20.018720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65437895c89743c1bf5980166acd9379)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:20.021568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:20.312345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65437895c89743c1bf5980166acd9379'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:20.313233 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65437895c89743c1bf5980166acd9379'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:20.314452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65437895c89743c1bf5980166acd9379'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:20.333583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65437895c89743c1bf5980166acd9379'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:20.444581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65437895c89743c1bf5980166acd9379'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:20.488369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1199/2400] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:19 2022] DELETE /identity/v3/users/65437895c89743c1bf5980166acd9379 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:20.494935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:20.495400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:20.495746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:20.552436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:20.554655 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:20.681804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1202/2401] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:20.692742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:20.694434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:20.840287 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a10fe-4fae-4def-89f8-48e305e9e9e3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:20.844262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:20.845075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:20.845587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:20.846081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:20.862912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:20.877978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:20.913090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1200/2402] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:21.153137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:21.384152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:21.385684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:21.479726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e3519-4c58-4a83-839e-64b2a401d1af None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:21.530566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:21.530566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:21.530566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:21.530566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:21.533665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:21.620839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:21.622454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:21.655571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2349d1-035e-4576-8509-69c179b5cf08 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:21.660941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:21.662446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:21.670188 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:21.670487 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:21.670728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:21.671958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] PATH_INFO: `/v3/projects/07776ffe18f540aaac15b1e39c0bf0b4` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:21.674294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07776ffe18f540aaac15b1e39c0bf0b4)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:21.675547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:21.685461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1203/2403] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:21.702398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:21.867944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07776ffe18f540aaac15b1e39c0bf0b4', 'request_id': 'req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:21.896493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07776ffe18f540aaac15b1e39c0bf0b4', 'request_id': 'req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:22.105917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07776ffe18f540aaac15b1e39c0bf0b4', 'request_id': 'req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:22.126874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] The token cache is being invalidate because project 07776ffe18f540aaac15b1e39c0bf0b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:22.127459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:22.128367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:22.131445 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1201/2404] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:21 2022] DELETE /identity/v3/projects/07776ffe18f540aaac15b1e39c0bf0b4 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:22.136306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:22.190784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666c9fb-4f9d-478c-a7ae-978b994ade49 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:22.284918 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:22.291337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:22.377573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:22.380325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:22.382807 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:22.383320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] PATH_INFO: `/v3/users/82f8f519d2114338bbf7a56147e4ef2e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:22.385360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f8f519d2114338bbf7a56147e4ef2e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:22.388490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:22.438245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb026f-3bbf-4c9f-ae99-d8247ba9ea14 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:22.446211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:22.447325 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:22.460514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:22.460514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] PATH_INFO: `/v3/users/1735d20129aa4b11bc8afc42340f6f5c` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:22.460514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1735d20129aa4b11bc8afc42340f6f5c)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:22.460514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:23.018632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f8f519d2114338bbf7a56147e4ef2e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.023425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f8f519d2114338bbf7a56147e4ef2e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.093545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1735d20129aa4b11bc8afc42340f6f5c'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.097909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1735d20129aa4b11bc8afc42340f6f5c'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.099376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1735d20129aa4b11bc8afc42340f6f5c'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.117908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1735d20129aa4b11bc8afc42340f6f5c'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.126575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f8f519d2114338bbf7a56147e4ef2e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.151058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f8f519d2114338bbf7a56147e4ef2e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.152010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f8f519d2114338bbf7a56147e4ef2e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.180712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1204/2405] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:22 2022] DELETE /identity/v3/users/82f8f519d2114338bbf7a56147e4ef2e => generated 0 bytes in 1044 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:23.182037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:23.184365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.219575 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d58435ba-0af1-4a11-8047-560042fda51d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1735d20129aa4b11bc8afc42340f6f5c'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.249153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1202/2406] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:22 2022] DELETE /identity/v3/users/1735d20129aa4b11bc8afc42340f6f5c => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:23.255534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.256196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.269265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.279425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a5e3e-0896-4995-87c0-e43fa1e193e7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:23.368635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:23.370232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.410044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:23.410993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.411344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.414945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] PATH_INFO: `/v3/users/293c3895eb9240f5aff70036301a5921` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.426615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=293c3895eb9240f5aff70036301a5921)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:23.426615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:23.466317 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1203/2407] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:23.486659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:23.492623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.607712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c405b4-d070-4097-98e9-46226c7c7e17 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:23.611781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:23.613253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.613734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.619553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.634100 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:23.636220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb83f1a-05f8-495b-8b31-89e650cc3e37 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:23.666092 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1204/2408] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:23.734871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293c3895eb9240f5aff70036301a5921'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.736577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293c3895eb9240f5aff70036301a5921'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.861450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293c3895eb9240f5aff70036301a5921'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.882201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293c3895eb9240f5aff70036301a5921'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.887499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293c3895eb9240f5aff70036301a5921'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:23.891099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:23.920010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1205/2409] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:23 2022] DELETE /identity/v3/users/293c3895eb9240f5aff70036301a5921 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:23.924174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.924471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.924758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.971206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:23.974178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.064497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:24.067260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.137503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1205/2410] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:24.145957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:24.157678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.196465 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477d4a5-2998-4902-a4b6-f5581e23da8b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:24.210684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:24.210684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.210684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.210684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] PATH_INFO: `/v3/projects/db0ab3e4124d402a8625e0648bc75857` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.210993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0ab3e4124d402a8625e0648bc75857)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:24.210993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:24.251314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994117-e21a-4e95-82e8-9ad8e0ab1b7c None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:24.258513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:24.259281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.259792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.260855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.273290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:24.275051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:24.286203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1206/2411] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:04:24.313373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:24.328493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.353035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a36ffd-cf1a-4754-a6f4-b8c602039c0c service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:24.355619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0ab3e4124d402a8625e0648bc75857', 'request_id': 'req-a87894c1-2179-4ab4-a18d-a9458ca4eed8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.372437 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0ab3e4124d402a8625e0648bc75857', 'request_id': 'req-a87894c1-2179-4ab4-a18d-a9458ca4eed8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.389432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0ab3e4124d402a8625e0648bc75857', 'request_id': 'req-a87894c1-2179-4ab4-a18d-a9458ca4eed8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.454376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:24.456030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.457007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.457517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] PATH_INFO: `/v3/projects/4a0086c45596414f95f854e8bc79bba0` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.459704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0086c45596414f95f854e8bc79bba0)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:24.464303 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:24.518998 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] The token cache is being invalidate because project db0ab3e4124d402a8625e0648bc75857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:24.519294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.521282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a87894c1-2179-4ab4-a18d-a9458ca4eed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.527123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1206/2412] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:24 2022] DELETE /identity/v3/projects/db0ab3e4124d402a8625e0648bc75857 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:24.529919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b1c1-8799-47f9-b6d5-844495e18ff3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.530218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b1c1-8799-47f9-b6d5-844495e18ff3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.530532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b1c1-8799-47f9-b6d5-844495e18ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.603137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:24.620703 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8998b1c1-8799-47f9-b6d5-844495e18ff3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:24.630724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8998b1c1-8799-47f9-b6d5-844495e18ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.656316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1207/2413] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:24.660272 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.661102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.661895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.691162 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0086c45596414f95f854e8bc79bba0', 'request_id': 'req-bd24bd24-f258-4906-a5c0-fda375466144'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.701245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:24.702823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.707996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0086c45596414f95f854e8bc79bba0', 'request_id': 'req-bd24bd24-f258-4906-a5c0-fda375466144'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.808759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0086c45596414f95f854e8bc79bba0', 'request_id': 'req-bd24bd24-f258-4906-a5c0-fda375466144'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.843503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1208/2414] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:24.844885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:24.850079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] The token cache is being invalidate because project 4a0086c45596414f95f854e8bc79bba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:24.850453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.852330 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd24bd24-f258-4906-a5c0-fda375466144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:24.857990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1207/2415] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:24 2022] DELETE /identity/v3/projects/4a0086c45596414f95f854e8bc79bba0 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:24.865181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8034aaf-ba3a-4a6b-aed2-c84089b7476f None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:24.873259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:24.874565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.875278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.875765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.876945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:24.877424 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:24.879989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505ab9d-c25f-442c-a907-50902b800cfb None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.879989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505ab9d-c25f-442c-a907-50902b800cfb None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.879989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505ab9d-c25f-442c-a907-50902b800cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.880485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:24.897305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1209/2416] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:24.901903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:24.907295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.943837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1505ab9d-c25f-442c-a907-50902b800cfb None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:24.943837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1505ab9d-c25f-442c-a907-50902b800cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:24.950328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89555aa-481f-4d1e-9b58-6999e03dd7e3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:24.968425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1208/2417] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:24.977345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.977701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.977992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.980946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:24.981624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:24.981915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:24.982194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:24.982906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:24.999028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.030932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:25.041089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:25.067306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1209/2418] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:25.072285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.088366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe174730-6796-440b-8e67-c83ba1a3917a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.096473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.097654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.098113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.098585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.099597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:25.100060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.102816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.123573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1210/2419] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:25.131523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.156112 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8ceeb-e644-4e93-9bb1-f7cfac622ed1 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.177286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.177286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.177286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.177609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1210/2420] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:24 2022] POST /identity/v3/projects => generated 365 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:25.178049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.178049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.181084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.181573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.254429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:25.296068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1211/2421] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:25 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:25.303425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.309958 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:25.341130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ffd25-7138-411d-8a41-d2daabc393f4 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.419529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.423726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.423726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.423726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.423726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.423726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.493455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e87666-b010-417b-b2e7-a930a029d697 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.503818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.504974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.504974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.509074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.517284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1211/2422] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:25 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:25.518944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.520841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.528666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.543587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c1c39-0c6e-4792-828b-be692506a24e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.559191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.560172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.560700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.561158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.562051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.564001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.582142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1212/2423] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:25.583484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.595332 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b86de-3301-4e0b-8982-1bf0293bbc4e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.601832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.602847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.607299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.607299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.607299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.609203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.634085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1213/2424] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:25.634085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.643503 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf545-a36e-4bc4-be46-2a446836f8f3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.648753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.649532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.649896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.650260 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] PATH_INFO: `/v3/projects/c94794f8585e4be7a0800a1d405e331d/users/9c528e777b114263aada0e5edf7019bb/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.676766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94794f8585e4be7a0800a1d405e331d, user_id=9c528e777b114263aada0e5edf7019bb, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.677429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1212/2425] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:25 2022] POST /identity/v3/users => generated 295 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:25.680672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.682624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.716313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cc2f-f0f6-488e-bf65-759fe2ac4697 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:25.729919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:25.730660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.731665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.731665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.731971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:25.733786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd7cbb-5b0a-4e07-8d33-f9202b25d855 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:25.749787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1213/2426] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:04:25.751483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402aed0-2ddb-4719-b033-d5057408986b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.751778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402aed0-2ddb-4719-b033-d5057408986b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.752065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402aed0-2ddb-4719-b033-d5057408986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.776109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5402aed0-2ddb-4719-b033-d5057408986b None None] MFA Rules not processed for user `6c67d6fcda3f4a938b2728f7ce2fc3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:25.777532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5402aed0-2ddb-4719-b033-d5057408986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:25.802708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1214/2427] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:25 2022] PUT /identity/v3/projects/c94794f8585e4be7a0800a1d405e331d/users/9c528e777b114263aada0e5edf7019bb/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:25.813927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:25.933773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1214/2428] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:25.933773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:25.933773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:25.933773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:25.986512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] MFA Rules not processed for user `9c528e777b114263aada0e5edf7019bb`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:25.988699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:26.010101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e234-1e64-4bc9-8a79-436447f38f30 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:26.026855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:26.028000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.028848 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.029425 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.030274 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:26.032091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:26.046536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1215/2429] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:26.047698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:26.050193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:26.076333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c7de3-e846-4e78-b0d7-50b4dfabce62 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:26.083426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:26.085386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.088902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.088902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] PATH_INFO: `/v3/projects/5f29656fa3a44cd8a2707a197a56dcd8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.088902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f29656fa3a44cd8a2707a197a56dcd8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:26.091909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:26.165022 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1215/2430] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:26.171503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:26.195007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c216a48-0ec2-4f90-b5e7-a6f2092f7fd1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:26.204678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:26.205633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.206420 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.207153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] PATH_INFO: `/v3/projects/4adf5b94a9a74dab9cc90772a1fe4b2d/users/52043d6970744c55a083f26a18dc6db6/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.240288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adf5b94a9a74dab9cc90772a1fe4b2d, user_id=52043d6970744c55a083f26a18dc6db6, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:26.248454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:26.364629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1216/2431] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:26 2022] PUT /identity/v3/projects/4adf5b94a9a74dab9cc90772a1fe4b2d/users/52043d6970744c55a083f26a18dc6db6/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:26.364629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:26.364629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:26.373481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f29656fa3a44cd8a2707a197a56dcd8', 'request_id': 'req-cc06467f-f35e-42a1-b782-b590249952af'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:26.388242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f29656fa3a44cd8a2707a197a56dcd8', 'request_id': 'req-cc06467f-f35e-42a1-b782-b590249952af'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:26.399783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f29656fa3a44cd8a2707a197a56dcd8', 'request_id': 'req-cc06467f-f35e-42a1-b782-b590249952af'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:26.508728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2657-b175-413b-9114-45c329a4b730 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:26.527135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:26.528115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.528728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.529173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.535654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] The token cache is being invalidate because project 5f29656fa3a44cd8a2707a197a56dcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:26.536179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:26.537483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-cc06467f-f35e-42a1-b782-b590249952af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:26.545495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1216/2432] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:26 2022] DELETE /identity/v3/projects/5f29656fa3a44cd8a2707a197a56dcd8 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:26.547233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c19146-b05c-4d13-9aa3-c4f850a5713e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.547673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c19146-b05c-4d13-9aa3-c4f850a5713e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.548618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c19146-b05c-4d13-9aa3-c4f850a5713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.666192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b8c19146-b05c-4d13-9aa3-c4f850a5713e None None] MFA Rules not processed for user `52043d6970744c55a083f26a18dc6db6`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:26.669313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b8c19146-b05c-4d13-9aa3-c4f850a5713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:26.687743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:26.692662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:26.697799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:26.721611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1217/2433] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:26.727989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:26.796601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1217/2434] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:26.798047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.798506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.798784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.857980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:26.859275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:26.909106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d9df3-c89d-48fc-bfb0-80c58818aa3d service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:26.919476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:26.920151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:26.920462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:26.920746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] PATH_INFO: `/v3/users/c87a083642834da6829614f3ca028e3b` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:26.929222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87a083642834da6829614f3ca028e3b)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:26.931268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:27.013950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1218/2435] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:04:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:27.034312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:27.039545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:27.065353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96425db5-1ae1-47ac-b1ce-c192b12bdedc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:27.070588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:27.071954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:27.073485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:27.073485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:27.101038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:27.105242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:27.129499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1219/2436] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:27.130499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:27.135302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:27.231103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87a083642834da6829614f3ca028e3b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:27.231103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87a083642834da6829614f3ca028e3b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:27.232337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87a083642834da6829614f3ca028e3b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:27.275534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed37e9-942e-425b-97a4-281339844908 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:27.286815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87a083642834da6829614f3ca028e3b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:27.294372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:27.297354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:27.299047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:27.299047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:27.300911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:27.319871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:27.332209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:27.335715 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:27.349842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1220/2437] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:04:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:04:27.364518 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87a083642834da6829614f3ca028e3b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:27.392568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1218/2438] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:26 2022] DELETE /identity/v3/users/c87a083642834da6829614f3ca028e3b => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:29.129266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:29.131022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:29.162327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b0c7-fda1-4bfa-b500-9cad658fe6ee service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:29.332019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:29.333837 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:29.353989 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:29.363646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:29.363646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:29.363646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] PATH_INFO: `/v3/projects/d55b6aef101947caa0160045d81b1193` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:29.363646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55b6aef101947caa0160045d81b1193)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:29.372106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:29.530091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958a6a-5e30-45e4-9fac-066006565005 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:29.542077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:29.542961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:29.543532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:29.543981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:29.551541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:29.661918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55b6aef101947caa0160045d81b1193', 'request_id': 'req-28161ec4-26ea-4241-8988-461d590a66c3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:29.683982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55b6aef101947caa0160045d81b1193', 'request_id': 'req-28161ec4-26ea-4241-8988-461d590a66c3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:29.707982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55b6aef101947caa0160045d81b1193', 'request_id': 'req-28161ec4-26ea-4241-8988-461d590a66c3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:29.747811 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:29.762872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:29.785079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1219/2439] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:29.846727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] The token cache is being invalidate because project d55b6aef101947caa0160045d81b1193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:29.847037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:29.847761 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-28161ec4-26ea-4241-8988-461d590a66c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:29.850744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1221/2440] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:29 2022] DELETE /identity/v3/projects/d55b6aef101947caa0160045d81b1193 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:29.854797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:29.869587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7884e6-4438-4222-9eed-57ca128935cb service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:29.983596 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:29.984785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:29.985144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:29.985959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] PATH_INFO: `/v3/users/a09e34d183c34dd6bfe2733e9abad20a` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:29.986829 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09e34d183c34dd6bfe2733e9abad20a)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:29.992593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:30.134841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:30.198060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:30.422978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09e34d183c34dd6bfe2733e9abad20a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:30.423899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09e34d183c34dd6bfe2733e9abad20a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:30.429293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09e34d183c34dd6bfe2733e9abad20a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:30.456803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09e34d183c34dd6bfe2733e9abad20a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:30.575899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09e34d183c34dd6bfe2733e9abad20a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:30.604139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1220/2441] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:29 2022] DELETE /identity/v3/users/a09e34d183c34dd6bfe2733e9abad20a => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:31.514696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5624-e6e5-4174-9b3c-3f5ed7b603f6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:31.515189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5624-e6e5-4174-9b3c-3f5ed7b603f6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:31.515502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f5624-e6e5-4174-9b3c-3f5ed7b603f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:31.563083 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6a0f5624-e6e5-4174-9b3c-3f5ed7b603f6 None None] MFA Rules not processed for user `78888cc145ba4052be05b1c321de44fd`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:31.564500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6a0f5624-e6e5-4174-9b3c-3f5ed7b603f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:31.692608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1222/2442] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:31.731077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:31.733396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:31.897724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed7b19-6c59-4cfd-a4b9-538350c06331 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:31.903873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:31.904862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:31.907467 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:31.907960 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] PATH_INFO: `/v3/projects/3250a40b6c3848309dad07640a66df93` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:31.909457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3250a40b6c3848309dad07640a66df93)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:31.911690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:32.140830 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3250a40b6c3848309dad07640a66df93', 'request_id': 'req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:32.167766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3250a40b6c3848309dad07640a66df93', 'request_id': 'req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:32.316116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3250a40b6c3848309dad07640a66df93', 'request_id': 'req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:32.329474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] The token cache is being invalidate because project 3250a40b6c3848309dad07640a66df93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:32.330477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:32.331240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2f26150b-a3dc-42a3-8abc-4fcc06d37a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:32.340572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:32.344476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:32.344476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:32.344665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1221/2443] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:31 2022] DELETE /identity/v3/projects/3250a40b6c3848309dad07640a66df93 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:32.370664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:32.380803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:32.383201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:32.557399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1223/2444] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:32.559709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:32.560124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:32.560667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:32.621887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:32.632395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:32.639051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1222/2445] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:32.643294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:32.666481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea353f4-a56c-46b9-9b37-84b7f59d48ae None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:32.667830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:32.668548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:32.668837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:32.669120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:32.671298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:32.673308 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:32.676383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:32.715485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1224/2446] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:04:32.716770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:32.727616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579badd-2de3-4ae9-a746-030a21c6d24f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:32.782497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:32.782497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:32.782497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:32.782497 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:32.784486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:32.784486 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:33.237978 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1223/2447] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:32 2022] POST /identity/v3/projects => generated 379 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:33.243208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:33.256044 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da62cd-d7c1-4770-810c-0065dd9ee517 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:33.379537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:33.380208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:33.380499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:33.380787 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:33.382089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:33.387034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:33.547252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1225/2448] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:33 2022] POST /identity/v3/users => generated 290 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:33.551678 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:33.566313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3a22c-e03a-4c3e-b416-a245748aebac admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:33.573582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:33.574220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:33.574540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:33.574822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:33.575563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:33.577221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:33.597934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1224/2449] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:04:33.599378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:33.617576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82cc82-131f-4e5e-b887-f83ca6085d80 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:33.625392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:33.626137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:33.626537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:33.626919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:33.627851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:33.629766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8934178-a492-4b61-86b0-97d8827b33f6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:33.650451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1226/2450] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:33.654073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:33.669014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb0775-e435-4e2a-97e7-55220d866a58 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:33.674374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:33.678948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:33.678948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:33.678948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] PATH_INFO: `/v3/projects/a1b1e6abd55a4b17b4a452a80a6bbdbc/users/73edfd05da254457b6ad9c9c9f662e4c/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:33.699652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b1e6abd55a4b17b4a452a80a6bbdbc, user_id=73edfd05da254457b6ad9c9c9f662e4c, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:33.702985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:33.776571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1225/2451] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:33 2022] PUT /identity/v3/projects/a1b1e6abd55a4b17b4a452a80a6bbdbc/users/73edfd05da254457b6ad9c9c9f662e4c/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:33.778689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:33.779618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:33.792145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:33.827168 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:33.828379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:33.930902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1227/2452] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:33.942387 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:33.946263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:34.065283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf431a-f3e5-4ebd-be53-d4d8255b79d2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:34.073261 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:34.074076 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:34.074509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:34.074888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:34.091701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:34.093705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816f9d5-cb21-4509-a49a-e1be98be3c54 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:34.107213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1226/2453] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:34.203040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:35.189457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:35.375109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:36.664939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:36.724498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:39.070671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:39.072278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:39.094780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1502e43-a888-42e4-8c77-c2494970e5d6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:39.101947 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:39.102926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:39.103917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:39.104372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] PATH_INFO: `/v3/users/195fc3305bc84c8f9afe2fda85130c9a` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:39.105893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=195fc3305bc84c8f9afe2fda85130c9a)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:39.108047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:39.331895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '195fc3305bc84c8f9afe2fda85130c9a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:39.332979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '195fc3305bc84c8f9afe2fda85130c9a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:39.414804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '195fc3305bc84c8f9afe2fda85130c9a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:39.441901 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '195fc3305bc84c8f9afe2fda85130c9a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:39.444659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '195fc3305bc84c8f9afe2fda85130c9a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:39.475623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1228/2454] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:39 2022] DELETE /identity/v3/users/195fc3305bc84c8f9afe2fda85130c9a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:39.479820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e7d9-52f8-4fd3-b0f1-cad055074ac4 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:39.480773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e7d9-52f8-4fd3-b0f1-cad055074ac4 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:39.482001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e7d9-52f8-4fd3-b0f1-cad055074ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:39.511533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6869e7d9-52f8-4fd3-b0f1-cad055074ac4 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:39.514334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6869e7d9-52f8-4fd3-b0f1-cad055074ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:39.683294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:39.745542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1227/2455] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:39.770084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:39.781489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:39.791433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:39.937478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304be2-d73a-48a3-94a7-71fe2af68a72 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:39.950379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:39.951103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:39.951982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:39.952301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:39.978124 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:39.979573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:40.006623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1229/2456] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:40.727716 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:40.799527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eeb41-207c-4291-999c-481d9350387d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:40.800173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eeb41-207c-4291-999c-481d9350387d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:40.800556 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eeb41-207c-4291-999c-481d9350387d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:40.837624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-694eeb41-207c-4291-999c-481d9350387d None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:40.839050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-694eeb41-207c-4291-999c-481d9350387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:40.915880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:40.919232 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:40.939615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6d1a-c7b8-4a91-92a0-2d6c8db3b89d service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:40.946466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:40.952053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:40.952497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:40.956575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] PATH_INFO: `/v3/projects/2d2056635d6a41f0a5a02c1dcb27baa8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:40.956575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2056635d6a41f0a5a02c1dcb27baa8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:40.956575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:41.018790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1228/2457] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:41.055236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eeb41-207c-4291-999c-481d9350387d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:41.055236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-694eeb41-207c-4291-999c-481d9350387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:41.161731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2056635d6a41f0a5a02c1dcb27baa8', 'request_id': 'req-e6a28b25-b326-43a1-9dc3-482697996094'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:41.183131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2056635d6a41f0a5a02c1dcb27baa8', 'request_id': 'req-e6a28b25-b326-43a1-9dc3-482697996094'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:41.210760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2056635d6a41f0a5a02c1dcb27baa8', 'request_id': 'req-e6a28b25-b326-43a1-9dc3-482697996094'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:41.280295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eeb41-207c-4291-999c-481d9350387d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:41.385714 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] The token cache is being invalidate because project 2d2056635d6a41f0a5a02c1dcb27baa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:41.386201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:41.388460 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:41.400956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1230/2458] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:40 2022] DELETE /identity/v3/projects/2d2056635d6a41f0a5a02c1dcb27baa8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:41.423695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:41.435459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:41.479302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:41.480214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:41.483752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:41.491966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:41.499998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:41.617439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:41.620199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:41.630116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1229/2459] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:04:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:04:41.631720 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28b25-b326-43a1-9dc3-482697996094 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:41.644788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:41.644788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:41.644788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:41.644788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] PATH_INFO: `/v3/users/95bf50a5409b4e9da6654913db2dfdbd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:41.648169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95bf50a5409b4e9da6654913db2dfdbd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:41.658318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:42.085187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95bf50a5409b4e9da6654913db2dfdbd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:42.090169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95bf50a5409b4e9da6654913db2dfdbd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:42.239649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95bf50a5409b4e9da6654913db2dfdbd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:42.264320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95bf50a5409b4e9da6654913db2dfdbd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:42.269129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95bf50a5409b4e9da6654913db2dfdbd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:42.297966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1231/2460] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:41 2022] DELETE /identity/v3/users/95bf50a5409b4e9da6654913db2dfdbd => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:42.644013 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:43.081840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:43.720263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:43.735911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c11e-2784-453c-bc8e-4dc51fff5c69 service swift] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:43.941443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:43.942103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:43.942418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:43.942706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] PATH_INFO: `/v3/projects/09c20714f0d44e788c7e12b31b7411e7` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:43.943972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c20714f0d44e788c7e12b31b7411e7)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:43.945884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:44.107675 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c20714f0d44e788c7e12b31b7411e7', 'request_id': 'req-8a87088c-6e0d-4904-9602-4e0ada27926a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.113548 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c20714f0d44e788c7e12b31b7411e7', 'request_id': 'req-8a87088c-6e0d-4904-9602-4e0ada27926a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.269045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c20714f0d44e788c7e12b31b7411e7', 'request_id': 'req-8a87088c-6e0d-4904-9602-4e0ada27926a'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.289241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] The token cache is being invalidate because project 09c20714f0d44e788c7e12b31b7411e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:44.289537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.303517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.305639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1230/2461] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:43 2022] DELETE /identity/v3/projects/09c20714f0d44e788c7e12b31b7411e7 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:44.312632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:44.318144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:44.339554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dee0-8432-45a7-955e-e0b619e23538 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:44.485571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:44.486266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:44.486608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:44.486981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] PATH_INFO: `/v3/users/2ece22462b4a46579c2559c8e06fd394` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:44.493658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ece22462b4a46579c2559c8e06fd394)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:44.496143 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:44.747966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ece22462b4a46579c2559c8e06fd394'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.749149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ece22462b4a46579c2559c8e06fd394'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.827070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ece22462b4a46579c2559c8e06fd394'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.855744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ece22462b4a46579c2559c8e06fd394'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.856810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-90fd603c-1e4a-4e29-9d70-4d524b401f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ece22462b4a46579c2559c8e06fd394'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:44.873939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1232/2462] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:44 2022] DELETE /identity/v3/users/2ece22462b4a46579c2559c8e06fd394 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:45.114168 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:46.027662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:46.029549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:46.049920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87088c-6e0d-4904-9602-4e0ada27926a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:46.123008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:46.125466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:46.125466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:46.125466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] PATH_INFO: `/v3/projects/37d87044285a400eab159d905d21b58a` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:46.126110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d87044285a400eab159d905d21b58a)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:46.128294 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:46.306230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d87044285a400eab159d905d21b58a', 'request_id': 'req-576f5889-bb57-4403-8565-d3cf5294486f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:46.318283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:46.324017 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d87044285a400eab159d905d21b58a', 'request_id': 'req-576f5889-bb57-4403-8565-d3cf5294486f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:46.386222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d87044285a400eab159d905d21b58a', 'request_id': 'req-576f5889-bb57-4403-8565-d3cf5294486f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:46.411232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] The token cache is being invalidate because project 37d87044285a400eab159d905d21b58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:04:46.411536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:46.422964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-576f5889-bb57-4403-8565-d3cf5294486f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:46.425353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1231/2463] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:46 2022] DELETE /identity/v3/projects/37d87044285a400eab159d905d21b58a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:46.432279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:46.432579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:46.438327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:46.482620 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:46.484291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:46.593443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1233/2464] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:46.595025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:46.595532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:46.595877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:46.637006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:46.637469 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:46.656978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:46.658433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1232/2465] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:46.694726 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b37513-dd02-4b6e-a8f9-00c0caebef4d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:46.706920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:46.707751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:46.708212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:46.715921 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:46.717148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:04:46.727855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:46.727855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:46.763072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:46.771245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1234/2466] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:04:46.778849 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77ac8b-3764-461f-9318-5c3017633831 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:46.829694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399211-0544-42c3-b92e-325257701011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:46.830883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399211-0544-42c3-b92e-325257701011 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:46.831280 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399211-0544-42c3-b92e-325257701011 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:46.831652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de399211-0544-42c3-b92e-325257701011 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:46.832659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399211-0544-42c3-b92e-325257701011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:46.839969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de399211-0544-42c3-b92e-325257701011 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.004735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.016253 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1233/2467] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:46 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:47.018565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94b25-a6e2-43b3-a113-718311c6392a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.195636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.196347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.196694 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.197598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.198405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.201046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.347158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1235/2468] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:47 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:47.351783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399211-0544-42c3-b92e-325257701011 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.363549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de399211-0544-42c3-b92e-325257701011 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.371227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.371227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.371227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.371227 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.371785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.373762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.386205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1234/2469] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:47.391561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.404762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfaf6e-39ed-42c9-b3da-b1631447f2ad admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.410252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.410970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.411254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.411529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.412894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.414855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5a3a7-a24e-49f0-ad57-f2d8c5df2f22 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.432815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1236/2470] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:47.434618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.453572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5c45e-0f5f-48e1-b7c6-f9a36d3cd84f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.469606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.470598 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.470892 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.471175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] PATH_INFO: `/v3/projects/45d5ca3652824227b24409f91f2d8a30/users/ad1b22c9cbfa4b9d9957370c38183ce4/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.502571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d5ca3652824227b24409f91f2d8a30, user_id=ad1b22c9cbfa4b9d9957370c38183ce4, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.505825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.517957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:47.610175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1235/2471] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:47 2022] PUT /identity/v3/projects/45d5ca3652824227b24409f91f2d8a30/users/ad1b22c9cbfa4b9d9957370c38183ce4/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:47.623430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a23d6a-0726-4dda-94de-5f34bec05622 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.623721 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a23d6a-0726-4dda-94de-5f34bec05622 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.624026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a23d6a-0726-4dda-94de-5f34bec05622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.656839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c3a23d6a-0726-4dda-94de-5f34bec05622 None None] MFA Rules not processed for user `ad1b22c9cbfa4b9d9957370c38183ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:47.658047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c3a23d6a-0726-4dda-94de-5f34bec05622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:47.737529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1237/2472] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:47.751636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.753723 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:47.825515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaae30-f9c7-4565-9d9b-3b9003951bbb admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.832360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.832897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.833221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.840260 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.853471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.855046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.866307 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1236/2473] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:48.039017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:48.384846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:48.385448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:48.385448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:48.438562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] MFA Rules not processed for user `73edfd05da254457b6ad9c9c9f662e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:48.440223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:48.443440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:48.669825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1238/2474] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:48.675024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:48.676987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:48.852748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a6c9f-c990-4279-bc3a-29f93ebb40f8 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:48.858783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:48.860237 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:48.860760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:48.861285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:48.863210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:48.867660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.048211 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1237/2475] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:04:48 2022] POST /identity/v3/projects => generated 379 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:49.049450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:49.066108 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93a059-c187-4635-8442-5ff4b884add4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:49.173189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:49.173950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:49.174294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:49.174635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:49.175353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:49.178304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.321072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1239/2476] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:04:49 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:04:49.327726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:49.347349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c80186-60b6-4304-a7dc-c0c2346bfdb2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:49.358846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:49.359753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:49.360245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:49.360729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:49.375364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:49.375364 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.402803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1238/2477] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:49.406868 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:49.421310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24f7a6-5a88-4401-b7d8-fc5c63663ca3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:49.428139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:49.429135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:49.429654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:49.430161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] PATH_INFO: `/v3/projects/9f223cbeefe149efb4ecbb07888c927e/users/ffaec5dad26d4d0b9625eaf238243cd4/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:49.474070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f223cbeefe149efb4ecbb07888c927e, user_id=ffaec5dad26d4d0b9625eaf238243cd4, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:49.483072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.577598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1240/2478] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:49 2022] PUT /identity/v3/projects/9f223cbeefe149efb4ecbb07888c927e/users/ffaec5dad26d4d0b9625eaf238243cd4/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:04:49.581896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:49.604661 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4cf6a-4eea-49ec-a3bc-ad001c2a2669 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:49.778399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:49.783033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:49.783033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:49.783033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:49.783033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:49.784032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:49.787246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.823708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1239/2479] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:04:49.825137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:49.846943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0113-50b5-4553-b921-ed3382eae886 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:49.873467 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:49.881211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:49.881211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:49.881211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] PATH_INFO: `/v3/projects/9f223cbeefe149efb4ecbb07888c927e/users/ffaec5dad26d4d0b9625eaf238243cd4/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:49.883434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f223cbeefe149efb4ecbb07888c927e, user_id=ffaec5dad26d4d0b9625eaf238243cd4, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:49.891100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d165-aa98-4e9a-87fe-bac6e8b4cb9d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:49.900786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:49.920244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1241/2480] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:04:49 2022] PUT /identity/v3/projects/9f223cbeefe149efb4ecbb07888c927e/users/ffaec5dad26d4d0b9625eaf238243cd4/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:51.129640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:51.564235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:53.664628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:53.838059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:55.207142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:55.685439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:56.249644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:56.925631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:56.946751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:56.976869 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b5de-0816-4e35-8fae-82d133939138 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:57.088528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:57.137955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:57.138724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:57.142768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:57.143522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] PATH_INFO: `/v3/users/52043d6970744c55a083f26a18dc6db6` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:57.146111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52043d6970744c55a083f26a18dc6db6)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:57.165069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:57.391534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52043d6970744c55a083f26a18dc6db6'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:57.395069 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52043d6970744c55a083f26a18dc6db6'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:57.397624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52043d6970744c55a083f26a18dc6db6'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:57.434740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52043d6970744c55a083f26a18dc6db6'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:57.506797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52043d6970744c55a083f26a18dc6db6'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:57.530044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1240/2481] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:56 2022] DELETE /identity/v3/users/52043d6970744c55a083f26a18dc6db6 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:04:57.534794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:57.535080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:57.535360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:57.577944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:57.579292 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:57.746870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1242/2482] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:57.773435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:57.775065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:57.903637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b884e7-5cab-46b7-9156-cfb93aff89b3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:57.922805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:57.923605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:57.924151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:57.924810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:57.950551 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:57.953924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:57.957171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:57.997309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1241/2483] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:58.495822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:59.649547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:59.730699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:59.732939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:59.749148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbbb5e-5811-45c3-af28-6858da3936b6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:59.761946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:59.762543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:59.763135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:59.763582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] PATH_INFO: `/v3/projects/4adf5b94a9a74dab9cc90772a1fe4b2d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:59.781639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4adf5b94a9a74dab9cc90772a1fe4b2d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:59.781639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:59.970788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4adf5b94a9a74dab9cc90772a1fe4b2d', 'request_id': 'req-4d51f09c-42be-494b-8904-b3c94a3d7238'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:04:59.979123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:59.982817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:59.984892 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4adf5b94a9a74dab9cc90772a1fe4b2d', 'request_id': 'req-4d51f09c-42be-494b-8904-b3c94a3d7238'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.011855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4adf5b94a9a74dab9cc90772a1fe4b2d', 'request_id': 'req-4d51f09c-42be-494b-8904-b3c94a3d7238'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.019614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cd28f-c827-46b0-954b-5949a573b65c service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:00.101468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] The token cache is being invalidate because project 4adf5b94a9a74dab9cc90772a1fe4b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:05:00.101775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.102866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.104744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1243/2484] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:04:59 2022] DELETE /identity/v3/projects/4adf5b94a9a74dab9cc90772a1fe4b2d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:00.272691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:00.299116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:00.300872 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:00.311528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:00.312833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] PATH_INFO: `/v3/users/ad1b22c9cbfa4b9d9957370c38183ce4` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:00.317561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1b22c9cbfa4b9d9957370c38183ce4)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:00.320529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:00.449538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:00.449538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:00.683063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1b22c9cbfa4b9d9957370c38183ce4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.685026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1b22c9cbfa4b9d9957370c38183ce4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.688565 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1b22c9cbfa4b9d9957370c38183ce4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.715631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51f09c-42be-494b-8904-b3c94a3d7238 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:00.745103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1b22c9cbfa4b9d9957370c38183ce4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.753176 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:00.755621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:00.756826 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:00.758600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:00.776738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:00.815827 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:00.821477 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581af46-22e3-4cdf-938e-a0b8aa692b5a service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:00.902940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1b22c9cbfa4b9d9957370c38183ce4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:00.903462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1244/2485] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:00.968716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1242/2486] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:04:59 2022] DELETE /identity/v3/users/ad1b22c9cbfa4b9d9957370c38183ce4 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:00.974206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68e61-773e-4dd1-99f9-2f98697501c2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:00.974541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68e61-773e-4dd1-99f9-2f98697501c2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:00.974861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68e61-773e-4dd1-99f9-2f98697501c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:01.087580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a1b68e61-773e-4dd1-99f9-2f98697501c2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:01.088835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a1b68e61-773e-4dd1-99f9-2f98697501c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:01.324535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:01.337074 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1245/2487] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:01.430658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:01.433610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:01.588751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:01.698862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c8416-988e-47a0-bcf9-0eced359cb52 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:01.711976 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:01.714659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:01.715759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:01.715759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:01.765606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:01.767343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:01.809566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1243/2488] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:02.451622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:02.459748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:02.459748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:02.508457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] MFA Rules not processed for user `ffaec5dad26d4d0b9625eaf238243cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:02.510062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.658207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1246/2489] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:02.678389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:02.680566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.855943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9859-7566-4107-91ec-5f3fa848e2c5 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:02.865555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:02.866657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:02.869406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:02.869406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:02.869797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.929351 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:02.930976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:03.010014 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b176a8-3f90-4c4f-9cff-0c181e812e2d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:03.019355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:03.021063 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:03.039053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:03.042534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:03.043427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195511e4-f457-44a6-b03f-0e3ccea43de6 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:03.043862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] PATH_INFO: `/v3/projects/45d5ca3652824227b24409f91f2d8a30` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:03.046089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d5ca3652824227b24409f91f2d8a30)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:03.049247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:03.073952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1244/2490] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:03.314732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d5ca3652824227b24409f91f2d8a30', 'request_id': 'req-735493c9-8043-4a66-bf06-886068697b20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:03.329077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d5ca3652824227b24409f91f2d8a30', 'request_id': 'req-735493c9-8043-4a66-bf06-886068697b20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:03.350172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d5ca3652824227b24409f91f2d8a30', 'request_id': 'req-735493c9-8043-4a66-bf06-886068697b20'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:03.466454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] The token cache is being invalidate because project 45d5ca3652824227b24409f91f2d8a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:05:03.470584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:03.470584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-735493c9-8043-4a66-bf06-886068697b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:03.503631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1247/2491] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:05:02 2022] DELETE /identity/v3/projects/45d5ca3652824227b24409f91f2d8a30 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:03.505981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:03.507066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:03.508285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:03.557853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:03.563670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:03.797431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1245/2492] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:03.806027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:03.806326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:03.807387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:03.842632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:03.858563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:03.860557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:03.884342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1248/2493] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:03.888921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:03.901133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afe904-ba62-428e-8945-bc0a07134a0d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:03.913828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:03.917431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:03.917431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:03.917431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:03.917431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:05:03.918375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:03.921600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:03.931653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:03.945067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1246/2494] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:05:03.958691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:03.965495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b1652-16cc-42ef-a102-797bf23804f2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:03.984279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:03.984943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:03.985235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:03.985518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:03.986204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:03.988594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:04.212619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1249/2495] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:05:03 2022] POST /identity/v3/projects => generated 371 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:05:04.212850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:04.224694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ab4c7-fdd5-4fac-b3e2-ff4d7b2fe922 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:04.416229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:04.417312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:04.417652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:04.417944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:04.418782 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:04.424004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:04.529435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1247/2496] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:05:04 2022] POST /identity/v3/users => generated 286 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:05:04.533207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:04.547110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea9d2-287e-4f76-ac54-df4c9903cc7e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:04.560993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:04.561649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:04.562102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:04.562427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:04.563314 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:04.566646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:04.584525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1250/2497] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:05:04.588652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:04.632972 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f36a2-32d6-42c9-ba19-46c92bcaeb1b admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:04.638102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:04.638732 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:04.639021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:04.639302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:04.640014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:04.641942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6a5f0-43f7-4be6-8f4c-740e7dfd8b52 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:04.671014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1248/2498] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:05:04.675385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:04.689438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e265-ad74-458a-b405-cfadfd7c862e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:04.699564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:04.699564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:04.699564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:04.699564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] PATH_INFO: `/v3/projects/804411d08ada467c916025fbfd35f940/users/4fd8a852ff504e06a35ea8fb6f2ddbf7/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:04.716209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804411d08ada467c916025fbfd35f940, user_id=4fd8a852ff504e06a35ea8fb6f2ddbf7, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:04.719284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43031767-0a90-4aef-890c-7c8c80765f41 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:04.867404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1251/2499] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:05:04 2022] PUT /identity/v3/projects/804411d08ada467c916025fbfd35f940/users/4fd8a852ff504e06a35ea8fb6f2ddbf7/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:04.879047 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332e87c-45ba-49c5-85f4-48bcbc0c221f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:04.880380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332e87c-45ba-49c5-85f4-48bcbc0c221f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:04.963240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332e87c-45ba-49c5-85f4-48bcbc0c221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:04.963240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0332e87c-45ba-49c5-85f4-48bcbc0c221f None None] MFA Rules not processed for user `4fd8a852ff504e06a35ea8fb6f2ddbf7`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:04.963240 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0332e87c-45ba-49c5-85f4-48bcbc0c221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:05.089966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1249/2500] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:06.266197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:06.266896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:06.266896 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:06.268436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1252/2501] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:06.284024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:06.288515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:06.288515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:06.337577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:06.347439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:06.554693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:06.565413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1250/2502] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:05:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:06.739966 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:06.742471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:06.887240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02d5a-223c-4975-bc80-7ebe8cac7d09 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:06.899982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:06.904689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:06.904689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:06.904689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:06.913995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:06.922712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c93be-df58-40e7-99af-361ca5c26905 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:06.935831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1253/2503] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:06.985589 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:07.009492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c19d7c-4646-42c6-afdb-a3b43c6c7394 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:07.032045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:07.032201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.033236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.033236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.050515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:07.052687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93f812-1e59-44ea-8aff-58748559ff27 service glance] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:07.074195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1251/2504] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:07.388470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.389079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.389079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.390602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1254/2505] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:07.404804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb570d-27f3-4a8c-9101-23d0a4dac285 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.405903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb570d-27f3-4a8c-9101-23d0a4dac285 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.406510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb570d-27f3-4a8c-9101-23d0a4dac285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.444278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-26eb570d-27f3-4a8c-9101-23d0a4dac285 None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:07.445552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-26eb570d-27f3-4a8c-9101-23d0a4dac285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:07.511244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1252/2506] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:05:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:07.525982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:07.528431 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:07.548235 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1849a9-c9de-4036-90c0-177a30603571 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:07.555870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:07.559978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.559978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.559978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.570818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:07.572707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ff0f4-d389-41fa-b05e-1a69cc1c8a09 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:07.583961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1255/2507] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:07.686827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2e82-b083-49ea-bf3a-53fa308c9665 None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.686827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2e82-b083-49ea-bf3a-53fa308c9665 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.687175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2e82-b083-49ea-bf3a-53fa308c9665 None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.690944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1253/2508] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:05:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:07.710329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f9a1-8229-490b-a363-9ea7604e611c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.716479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f9a1-8229-490b-a363-9ea7604e611c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.716479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018f9a1-8229-490b-a363-9ea7604e611c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.746279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a018f9a1-8229-490b-a363-9ea7604e611c None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:07.747549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a018f9a1-8229-490b-a363-9ea7604e611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:07.764668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1256/2509] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:05:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:07.805913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed2e82-b083-49ea-bf3a-53fa308c9665 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:07.825855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed2e82-b083-49ea-bf3a-53fa308c9665 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:07.842605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:07.843474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:07.843920 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:07.845111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:07.864483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:07.867988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:07.884166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1254/2510] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:17.737128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:17.885776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=37 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:25.960669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779bdc0-f4ce-4e6b-8c92-2e3c61322c46 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:25.961330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779bdc0-f4ce-4e6b-8c92-2e3c61322c46 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:25.961330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779bdc0-f4ce-4e6b-8c92-2e3c61322c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:25.981943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f779bdc0-f4ce-4e6b-8c92-2e3c61322c46 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:25.983213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f779bdc0-f4ce-4e6b-8c92-2e3c61322c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:26.045027 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1257/2511] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:05:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:26.070241 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:26.071961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:26.144292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1043dd-736e-439d-9f51-873ed22f7678 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:26.149395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:26.150016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:26.150304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:26.150623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:26.158974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:26.161264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:26.169806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1255/2512] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:05:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 18:05:32.192113 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:32.192897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:32.192897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:32.213282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:32.214536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:32.222942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1258/2513] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:05:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:32.235527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:32.247018 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed506243-4358-4979-9420-a0d0dba0fd15 service swift] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:32.252555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:32.253522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:32.253781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:32.254160 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:32.263783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:32.265397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab6a60-2ae7-4d6c-9484-7bb384b00a21 service swift] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:32.275120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1256/2514] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:05:42.171121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:42.183291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895c20b-0a40-440b-8d00-76cec415bf7c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:42.188384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:42.189301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:42.189772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:42.190110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:42.192583 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:42.271472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=38 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:42.283178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:42.284919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:42.296550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1259/2515] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:50.553361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd037f6-b4b6-46c5-a063-48b6551569e9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:50.554302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd037f6-b4b6-46c5-a063-48b6551569e9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:50.554773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd037f6-b4b6-46c5-a063-48b6551569e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:50.579170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2bd037f6-b4b6-46c5-a063-48b6551569e9 None None] MFA Rules not processed for user `73edfd05da254457b6ad9c9c9f662e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:50.581866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2bd037f6-b4b6-46c5-a063-48b6551569e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:50.663406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1257/2516] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:52.313838 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=58 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:52.822485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:52.824268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:52.925056 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fdcf3-526c-45d0-8e8b-82c10235ab65 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:52.932605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:52.933370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:52.933663 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:52.933944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] PATH_INFO: `/v3/users/9c528e777b114263aada0e5edf7019bb` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:52.935549 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c528e777b114263aada0e5edf7019bb)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:52.937931 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:53.174287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c528e777b114263aada0e5edf7019bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:53.174287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c528e777b114263aada0e5edf7019bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:53.246891 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c528e777b114263aada0e5edf7019bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:53.286857 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c528e777b114263aada0e5edf7019bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:53.288263 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c528e777b114263aada0e5edf7019bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:53.304999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1260/2517] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:05:52 2022] DELETE /identity/v3/users/9c528e777b114263aada0e5edf7019bb => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:53.310008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f642-c52c-49b6-9b2d-758dd9a24b3b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:53.310634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f642-c52c-49b6-9b2d-758dd9a24b3b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:53.310739 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f642-c52c-49b6-9b2d-758dd9a24b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:53.332263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e489f642-c52c-49b6-9b2d-758dd9a24b3b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:53.333583 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e489f642-c52c-49b6-9b2d-758dd9a24b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:53.430278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1258/2518] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:53.444899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:53.446510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:53.512386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83c1dd-9b50-44f9-82ba-294eac7d2cc2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:53.516360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:53.517000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:53.517354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:53.517953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:53.527903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:53.529427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:53.538251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:53.538690 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:53.539072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:53.540775 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1261/2519] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:53.570900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:53.572302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:53.634819 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1259/2520] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:53.649740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:53.651756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:53.679326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:53.724977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7369-ad90-4be3-a8ef-e8d7101180ca service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:53.731448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:53.732916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:53.733258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:53.733542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:53.742772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:53.744373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7b659-3e68-4720-b576-a8361c5c2b4f service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:53.754128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1262/2521] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:05:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:05:53.963938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:53.965435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:53.992647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55db61-02e7-4021-93b9-18950c28beba None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:53.997758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:53.998440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.007180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.007180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] PATH_INFO: `/v3/projects/c94794f8585e4be7a0800a1d405e331d` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.007180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94794f8585e4be7a0800a1d405e331d)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.007180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.151768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94794f8585e4be7a0800a1d405e331d', 'request_id': 'req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:54.153427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:54.159379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94794f8585e4be7a0800a1d405e331d', 'request_id': 'req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:54.234196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94794f8585e4be7a0800a1d405e331d', 'request_id': 'req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:54.266676 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] The token cache is being invalidate because project c94794f8585e4be7a0800a1d405e331d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:05:54.267897 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:54.268742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-53fc9d19-ab9e-4e62-abd0-6813366fd1e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:54.274788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1260/2522] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:05:53 2022] DELETE /identity/v3/projects/c94794f8585e4be7a0800a1d405e331d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:54.276052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.276349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.276650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.299988 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:54.301751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:54.370198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1263/2523] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:54.378537 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.379470 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.379760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.422877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:54.425196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:54.434725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1261/2524] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:54.439265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.450598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8da9c-86e6-4d7e-a026-07f54d723a6b None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.455811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.456440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.456729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.457012 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.457786 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:05:54.458098 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.459811 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.481889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1264/2525] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:05:54.482677 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.491641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e24ef4-129f-4041-b1f3-512699834c3b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.496770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.497383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.497666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.497944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.498636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.501503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.623434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1262/2526] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:05:54 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:05:54.628225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.638264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398f12e-9852-40f0-b196-d18b583b10ff admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.709039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.709677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.709962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.710242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.715335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.717251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.871115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1265/2527] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:05:54 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:05:54.875549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.887847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f39b-6c49-4212-b19a-8f7483dd3ce6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.892881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.893759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.894179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.894624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.895631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.897693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.909878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1263/2528] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:05:54.914181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.924874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929e68-7716-47ce-a94d-468d70f62542 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.929500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.930148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.930946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.931428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.932158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.933765 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3f103-4d7d-41d9-ad7f-c70e80df028f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.947670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1266/2529] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:05:54.950313 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.960592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129fda5-fc31-479e-8f70-73de344f014c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.965382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.966035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.966300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.966623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] PATH_INFO: `/v3/projects/3c98157e4eb44d91ab6e16179d6dd9f1/users/e87cc4903ce246389ccef3d6dfffb904/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.992937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c98157e4eb44d91ab6e16179d6dd9f1, user_id=e87cc4903ce246389ccef3d6dfffb904, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.992937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:55.098319 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1264/2530] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:05:54 2022] PUT /identity/v3/projects/3c98157e4eb44d91ab6e16179d6dd9f1/users/e87cc4903ce246389ccef3d6dfffb904/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:05:55.108973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5ed4-05d7-437a-b23e-03459a940ab5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.109269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5ed4-05d7-437a-b23e-03459a940ab5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.109554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5ed4-05d7-437a-b23e-03459a940ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.140459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b05a5ed4-05d7-437a-b23e-03459a940ab5 None None] MFA Rules not processed for user `e87cc4903ce246389ccef3d6dfffb904`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:55.141758 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b05a5ed4-05d7-437a-b23e-03459a940ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:55.148592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:55.248788 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1267/2531] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:55.261036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:55.263061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:55.282240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba426-4676-49ba-935e-1844168838f2 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.282577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba426-4676-49ba-935e-1844168838f2 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.283421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba426-4676-49ba-935e-1844168838f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.305476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-dabba426-4676-49ba-935e-1844168838f2 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:55.306823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dabba426-4676-49ba-935e-1844168838f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:55.323504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7eb4-1734-485a-9d35-c692116204d2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:55.327456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:55.328079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.328366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.328648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.338902 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:55.340543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:55.350276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1265/2532] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:55.365001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1268/2533] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:55.376360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:55.379346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabba426-4676-49ba-935e-1844168838f2 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:55.380763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-dabba426-4676-49ba-935e-1844168838f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:55.441122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50c79e-0837-400c-b4de-6d83ce0410d7 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:55.445909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:55.447051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.447332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.447611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] PATH_INFO: `/v3/users/e87cc4903ce246389ccef3d6dfffb904` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.448928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87cc4903ce246389ccef3d6dfffb904)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:55.449439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabba426-4676-49ba-935e-1844168838f2 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:55.450923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:55.455953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:55.456591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.456879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.457162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.467439 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:55.467951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:55.469053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453de2-3f1a-4d28-af08-ae5037aaf2da service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:55.478858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1269/2534] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:05:55.711539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87cc4903ce246389ccef3d6dfffb904'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:55.712203 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87cc4903ce246389ccef3d6dfffb904'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:55.712816 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87cc4903ce246389ccef3d6dfffb904'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:55.724995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87cc4903ce246389ccef3d6dfffb904'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:55.844256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87cc4903ce246389ccef3d6dfffb904'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:55.858136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1266/2535] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:05:55 2022] DELETE /identity/v3/users/e87cc4903ce246389ccef3d6dfffb904 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:55.863498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:55.863790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:55.864071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:55.882853 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:55.884055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:55.922623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:55.943670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1270/2536] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:55.954040 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:55.955901 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:56.016042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037ea83-2cf4-47a9-ac25-2d9bca2759c3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:56.019800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:56.020463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.020749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.021029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.031800 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:56.033751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad80e79-3c42-4ec5-af27-331c96c809c0 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:56.043983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1267/2537] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:56.556949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:56.558562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:56.571650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef1248-948f-4bb2-b119-b73bb6afa2c1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:56.576828 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:56.579984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.579984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.579984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] PATH_INFO: `/v3/projects/3c98157e4eb44d91ab6e16179d6dd9f1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.579984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c98157e4eb44d91ab6e16179d6dd9f1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:56.581337 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:56.695536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c98157e4eb44d91ab6e16179d6dd9f1', 'request_id': 'req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:56.702524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c98157e4eb44d91ab6e16179d6dd9f1', 'request_id': 'req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:56.709110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c98157e4eb44d91ab6e16179d6dd9f1', 'request_id': 'req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:56.829279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] The token cache is being invalidate because project 3c98157e4eb44d91ab6e16179d6dd9f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:05:56.830201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:56.830980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8b1de89e-befe-4a0b-9e44-6ac3b2b958bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:05:56.832664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1271/2538] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:05:56 2022] DELETE /identity/v3/projects/3c98157e4eb44d91ab6e16179d6dd9f1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:05:56.837974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.838488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.838649 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.862474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:56.863697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:56.924534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1268/2539] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:56.933899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.934189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.934638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.955544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:56.956960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:56.966033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1272/2540] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:56.972242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:56.984163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce30f-9ce6-4b1b-a9f9-8dd85d94b260 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:56.989656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:56.990292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.990623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.990907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.991693 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:05:56.992003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:56.993720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.006635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1269/2541] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:05:57.011458 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.022830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed9cac-c6d8-41e9-8279-9dad2f64d269 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.027816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.028441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.028730 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.029011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.029693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.031733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.144409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1273/2542] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:05:57 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:05:57.148776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.159577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375090a-f9db-4c8d-b0d7-b2390361460c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.179363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:57.219509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.220136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.221053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.221393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.222083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.229746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.319851 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1270/2543] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:05:57 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:05:57.324166 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.335511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636af4c-0f1d-4d91-97b8-86963822a844 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.340453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.341090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.341380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.341665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.342499 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.344227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.354528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1274/2544] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:05:57.358706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.369120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c1b7-181e-4eac-bf1a-0b2c128145d2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.374638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.375366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.375652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.375932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.377142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.378798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcbdd6-4cfe-4ea3-a85a-30741898de2f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.390376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1271/2545] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:05:57.394764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.405211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8011dd-0e7d-4888-a8fc-2ac6703d5f7b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.410743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.411407 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.411698 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.411982 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] PATH_INFO: `/v3/projects/613c6d4f37814a209077ac2c19506674/users/aa7941170fe74f9c818f4ab612fca33f/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.427342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=613c6d4f37814a209077ac2c19506674, user_id=aa7941170fe74f9c818f4ab612fca33f, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.429978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.501752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1275/2546] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:05:57 2022] PUT /identity/v3/projects/613c6d4f37814a209077ac2c19506674/users/aa7941170fe74f9c818f4ab612fca33f/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:05:57.507078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a569ba-a5cd-4539-afbe-c68a15453fa9 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.507371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a569ba-a5cd-4539-afbe-c68a15453fa9 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.507653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a569ba-a5cd-4539-afbe-c68a15453fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.527506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-17a569ba-a5cd-4539-afbe-c68a15453fa9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:57.528711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-17a569ba-a5cd-4539-afbe-c68a15453fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:57.586631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1272/2547] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:57.596729 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.598619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:57.655996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea1a15-7ae9-4b77-b4fb-ea775d202bde admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.660349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.660973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.661306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.661574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.670636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.672090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.681745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1276/2548] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:59.271799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:59.272371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:59.272371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:59.304937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] MFA Rules not processed for user `aa7941170fe74f9c818f4ab612fca33f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:59.306189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:59.414876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1273/2549] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:59.426537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:59.428873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:59.437983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:59.509331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e92a5f-4496-4d9c-9c09-135296c7f9b3 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:59.528085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:59.529043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:59.529338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:59.529673 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:59.545615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:59.547710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:59.559278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1277/2550] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:00.394229 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:00.396123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:00.410695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129816-71e9-449d-a61b-d4a4b473b4bd None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:00.416264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:00.416922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:00.417210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:00.417489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] PATH_INFO: `/v3/users/6c67d6fcda3f4a938b2728f7ce2fc3ea` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:00.418715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c67d6fcda3f4a938b2728f7ce2fc3ea)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:00.420640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:00.665396 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:00.866683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c67d6fcda3f4a938b2728f7ce2fc3ea'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:00.867471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c67d6fcda3f4a938b2728f7ce2fc3ea'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:00.869224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c67d6fcda3f4a938b2728f7ce2fc3ea'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:00.891140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c67d6fcda3f4a938b2728f7ce2fc3ea'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.044468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-02eb02d1-c73e-48b9-8698-043ca073bbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c67d6fcda3f4a938b2728f7ce2fc3ea'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.062961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1274/2551] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:06:00 2022] DELETE /identity/v3/users/6c67d6fcda3f4a938b2728f7ce2fc3ea => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:01.373836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:01.387294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad212958-56a0-4106-a2b8-939c32840ab4 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:01.452189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:01.453152 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.453650 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.454160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] PATH_INFO: `/v3/projects/834ec9d4361246999cc4e822acaa6b77` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.456722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=834ec9d4361246999cc4e822acaa6b77)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:01.459323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:01.616100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834ec9d4361246999cc4e822acaa6b77', 'request_id': 'req-6f158f21-b8a5-4016-90ae-df37be38bd4b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.622558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834ec9d4361246999cc4e822acaa6b77', 'request_id': 'req-6f158f21-b8a5-4016-90ae-df37be38bd4b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.629109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834ec9d4361246999cc4e822acaa6b77', 'request_id': 'req-6f158f21-b8a5-4016-90ae-df37be38bd4b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.749980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] The token cache is being invalidate because project 834ec9d4361246999cc4e822acaa6b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:06:01.750512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.751182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6f158f21-b8a5-4016-90ae-df37be38bd4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:01.753441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1278/2552] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:06:01 2022] DELETE /identity/v3/projects/834ec9d4361246999cc4e822acaa6b77 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:01.759338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379da30-3459-4b36-8a91-cd0df8d64e05 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.760298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379da30-3459-4b36-8a91-cd0df8d64e05 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.760786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379da30-3459-4b36-8a91-cd0df8d64e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.784372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5379da30-3459-4b36-8a91-cd0df8d64e05 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:01.787861 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5379da30-3459-4b36-8a91-cd0df8d64e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:01.886683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.886683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.886683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.887035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1275/2553] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:01.887035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.887035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.887035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.903648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] MFA Rules not processed for user `aa7941170fe74f9c818f4ab612fca33f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:01.905051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:01.909903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:01.911268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:01.919886 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1276/2554] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:01.927171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:01.938026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504be0b-8295-4fc2-875b-595db3f4c88d None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:01.944318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:01.944995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.945687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.945687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.946320 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:06:01.946726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:01.948274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:01.948513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:01.961539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1277/2555] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:06:01.965966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:01.973531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1279/2556] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:01.977528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f8a3-204b-4506-a19e-06d26b5b2dbf admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:01.982190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:01.982823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:01.983106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:01.983380 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:01.984062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:01.986741 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.086404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1278/2557] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:06:01 2022] POST /identity/v3/projects => generated 399 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:06:02.090968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.103682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4afc-5e14-4487-ae17-094175bac4e1 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.162229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.163222 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.167300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.167300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.167300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.167300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.254150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1280/2558] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:06:02 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:06:02.255757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.266460 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd15b33-b9c3-4ef9-a694-106f59121b63 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.271733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.272344 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.272982 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.276744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.276744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.276939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.287528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1279/2559] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:02.291776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.303270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821f80d-ccc8-4de7-8df6-55a1053407b7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.308145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.308770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.309059 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.309343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.310049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.311685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1a9c-28e2-4adb-886e-571d426352cf admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.323225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1281/2560] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:02.327372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.338654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875749d3-c208-496d-8792-704d7f967f8d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.343572 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.344247 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.344549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.344835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] PATH_INFO: `/v3/projects/59a60575db5d4454b8404d3ce06dd5af/users/20f1890a1da34d7488f31b4a040dd58e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.361204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a60575db5d4454b8404d3ce06dd5af, user_id=20f1890a1da34d7488f31b4a040dd58e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.364472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.467631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1280/2561] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:06:02 2022] PUT /identity/v3/projects/59a60575db5d4454b8404d3ce06dd5af/users/20f1890a1da34d7488f31b4a040dd58e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:02.479835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.480296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.480519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.509784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] MFA Rules not processed for user `20f1890a1da34d7488f31b4a040dd58e`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:02.511026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:02.575617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1282/2562] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:02.587821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.589752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:02.666441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290a3a4-4686-40dd-8927-adf9faad7da5 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.670854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.675899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.675899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.675899 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.683832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.685536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d6808-2c1f-46eb-b08d-358eef9b449c service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.708600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1281/2563] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:02.851662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:04.418058 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:04.419522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:04.474570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:04.489832 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea75f2a-a506-4bcb-ad0e-b9efb526f0b4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:04.494440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:04.495093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:04.495394 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:04.495681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] PATH_INFO: `/v3/users/4fd8a852ff504e06a35ea8fb6f2ddbf7` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:04.496865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd8a852ff504e06a35ea8fb6f2ddbf7)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:04.498881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:04.726026 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd8a852ff504e06a35ea8fb6f2ddbf7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:04.726690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd8a852ff504e06a35ea8fb6f2ddbf7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:04.800734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd8a852ff504e06a35ea8fb6f2ddbf7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:04.813839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd8a852ff504e06a35ea8fb6f2ddbf7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:04.814646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd8a852ff504e06a35ea8fb6f2ddbf7'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:04.827136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1283/2564] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:06:04 2022] DELETE /identity/v3/users/4fd8a852ff504e06a35ea8fb6f2ddbf7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:04.831722 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:04.832174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:04.832343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:04.853617 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:04.854854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:04.915422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1282/2565] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:04.926531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:04.928006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:04.994216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df46b6c-e2df-4a54-83a1-8998f09020ed admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:04.998290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:04.999011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:04.999296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:04.999580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.022079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:05.026069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b62822-b2b0-4eac-8a91-afcf5216b665 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:05.034156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1284/2566] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:05.538322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:05.539907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.553172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2c05b-a88d-4e30-8260-92ab55857ad5 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:05.557888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:05.562873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.562873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.562873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] PATH_INFO: `/v3/projects/804411d08ada467c916025fbfd35f940` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.562873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804411d08ada467c916025fbfd35f940)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:05.562873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:05.681727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804411d08ada467c916025fbfd35f940', 'request_id': 'req-599ad4ac-d962-4d07-8523-fab6a35ac8c1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:05.687200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804411d08ada467c916025fbfd35f940', 'request_id': 'req-599ad4ac-d962-4d07-8523-fab6a35ac8c1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:05.759794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804411d08ada467c916025fbfd35f940', 'request_id': 'req-599ad4ac-d962-4d07-8523-fab6a35ac8c1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:05.771683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] The token cache is being invalidate because project 804411d08ada467c916025fbfd35f940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:06:05.771974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:05.772637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-599ad4ac-d962-4d07-8523-fab6a35ac8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:05.774137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1283/2567] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:06:05 2022] DELETE /identity/v3/projects/804411d08ada467c916025fbfd35f940 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:05.779515 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.779990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.780115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.801362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:05.802610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.865062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1285/2568] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:05.872940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.873407 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.873504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.894665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:05.895860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.903882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1284/2569] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:05.908946 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:05.918658 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474269c-26d7-45b6-ab60-1a52ddd2e6f7 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:05.923377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:05.923978 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.924268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.924543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.925240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:06:05.925544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:05.927185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:05.940918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:05.940918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1286/2570] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:06:05.943985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:05.953418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0005-2305-48c2-a7da-3074f9ee4fd1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:05.959388 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:05.963153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.963153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.963153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.963153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:05.963153 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.058888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1285/2571] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:06:05 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:06:06.063238 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:06.074295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558600eb-d26d-49db-b0e6-982555a1ab24 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:06.135538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:06.136231 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.136517 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.136797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.137462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:06.139675 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.244017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1287/2572] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:06:06 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:06:06.247195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:06.256029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c90a16-f7e0-456c-a5ff-70ea635886bc admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:06.260725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:06.261431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.261817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.262239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.262990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:06.264639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.276028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1286/2573] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:06.280136 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:06.291727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b377aa3-f98a-42ad-8378-1f331f87ac80 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:06.296711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:06.297344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.297627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.297930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.299052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:06.300676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc20737-1883-4591-8775-f463fa1df627 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.311612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1288/2574] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:06:06.316555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:06.326724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e1b7-45cb-45bb-b391-ff928ffc5ff8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:06.332226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:06.332917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.333244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.333530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] PATH_INFO: `/v3/projects/efa7968cb99d4470b48f8ff2fd8bad7a/users/186ea84ee37d48c186190438adce39b9/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.345852 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:06.349636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa7968cb99d4470b48f8ff2fd8bad7a, user_id=186ea84ee37d48c186190438adce39b9, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:06.352457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.449793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1287/2575] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:06:06 2022] PUT /identity/v3/projects/efa7968cb99d4470b48f8ff2fd8bad7a/users/186ea84ee37d48c186190438adce39b9/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:06:06.455784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac50c-d6af-4209-b487-0acb28b85376 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.458777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac50c-d6af-4209-b487-0acb28b85376 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.458777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac50c-d6af-4209-b487-0acb28b85376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.477302 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6f0ac50c-d6af-4209-b487-0acb28b85376 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:06.478624 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6f0ac50c-d6af-4209-b487-0acb28b85376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:06.537562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1289/2576] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:06.547275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:06.549023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:06.606045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96aff9-4f52-45fa-9176-ef38bd18f7d2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:06.609735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:06.610468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:06.610860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:06.611140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:06.621176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:06.622645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:06.632904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1288/2577] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:06:12.294960 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.296801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.297160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.344962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] MFA Rules not processed for user `186ea84ee37d48c186190438adce39b9`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:12.347248 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:12.437195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:12.445304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1290/2578] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:12.453463 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560aa2-ca18-46bc-b653-48bfb60e9f78 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:12.468721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:12.469367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.471351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.475334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.475334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:12.481118 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:12.622765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1289/2579] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:06:12 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:06:12.627441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:12.638474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026010bb-07b1-45ac-b2f4-ca0511d10391 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:12.643917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:12.702253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:12.702904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.703195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.703481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.704297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:12.706434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:12.789186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1291/2580] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:06:12 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:06:12.793110 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:12.805409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12d4e8-17f6-437d-a1a0-bcb85ee552cd admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:12.810173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:12.810746 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.811032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.811321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.812011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:12.814488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:12.826778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1290/2581] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:12.831687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:12.842224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f82384-1388-4233-8819-f4c9d63d16e1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:12.847500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:12.848188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.848480 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.854340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] PATH_INFO: `/v3/projects/a0a527edebc54195bb89297144e5cae4/users/10550bec9e91406880734d9543cfe8bd/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.865099 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a527edebc54195bb89297144e5cae4, user_id=10550bec9e91406880734d9543cfe8bd, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:12.868079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:12.959277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1292/2582] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:06:12 2022] PUT /identity/v3/projects/a0a527edebc54195bb89297144e5cae4/users/10550bec9e91406880734d9543cfe8bd/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:12.963336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:12.974543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f0ceb-bfc0-4bad-a258-49b2cbb1f9ff admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:13.055055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:13.055685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:13.055969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:13.056249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:13.056981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:13.058574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f6aeb-cd72-4d65-8696-15537815aa88 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:13.070625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1291/2583] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:13.074745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:13.085879 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8ee0f-d883-4491-abde-de737144691b admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:13.090870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:13.091558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:13.091872 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:13.092161 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] PATH_INFO: `/v3/projects/a0a527edebc54195bb89297144e5cae4/users/10550bec9e91406880734d9543cfe8bd/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:13.094189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a527edebc54195bb89297144e5cae4, user_id=10550bec9e91406880734d9543cfe8bd, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:13.097756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:13.109310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1293/2584] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:06:13 2022] PUT /identity/v3/projects/a0a527edebc54195bb89297144e5cae4/users/10550bec9e91406880734d9543cfe8bd/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:19.408564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a1f9-36ef-4ab2-8b82-58a51b4b6c0c None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:19.409116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a1f9-36ef-4ab2-8b82-58a51b4b6c0c None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:19.409116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a1f9-36ef-4ab2-8b82-58a51b4b6c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:19.448656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f383a1f9-36ef-4ab2-8b82-58a51b4b6c0c None None] MFA Rules not processed for user `10550bec9e91406880734d9543cfe8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:19.449893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f383a1f9-36ef-4ab2-8b82-58a51b4b6c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:19.499914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:19.527523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1292/2585] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:19.541606 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:19.543520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:19.626700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74870f-06fd-48e9-8e3e-92f1cdb8afd1 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:19.633034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:19.633781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:19.634199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:19.634621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:19.697728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:19.699510 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:19.703819 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:19.710313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1294/2586] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:22.891728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148009-a368-4708-ba80-5f13ea4d6006 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:22.892266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148009-a368-4708-ba80-5f13ea4d6006 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:22.892480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65148009-a368-4708-ba80-5f13ea4d6006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:22.915653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-65148009-a368-4708-ba80-5f13ea4d6006 None None] MFA Rules not processed for user `186ea84ee37d48c186190438adce39b9`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:22.916916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-65148009-a368-4708-ba80-5f13ea4d6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:22.924482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1293/2587] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:25.450393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:25.451884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:25.555710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45978c-76d7-45c4-afc9-d81a2651a51c service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:25.563265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:25.564080 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:25.564376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:25.564683 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] PATH_INFO: `/v3/users/20f1890a1da34d7488f31b4a040dd58e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:25.566123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f1890a1da34d7488f31b4a040dd58e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:25.568121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:25.861890 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f1890a1da34d7488f31b4a040dd58e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:25.862911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f1890a1da34d7488f31b4a040dd58e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:25.976343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f1890a1da34d7488f31b4a040dd58e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:25.994114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f1890a1da34d7488f31b4a040dd58e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:25.995035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f1890a1da34d7488f31b4a040dd58e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:26.011220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1295/2588] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:06:25 2022] DELETE /identity/v3/users/20f1890a1da34d7488f31b4a040dd58e => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:26.016402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:26.016903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:26.017200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:26.048536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:26.055561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:26.132437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1294/2589] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:26.139784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:26.144276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:26.223509 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2649d49-3afb-408d-8ef9-c7005be05c45 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:26.227942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:26.228257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:26.228523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:26.228802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:26.240568 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:26.242043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:26.253816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1296/2590] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:26.915990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:26.917453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:26.932694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb25c4-cfb6-47e6-88d6-cfa02bb3047e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:26.941561 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:26.948491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:26.948915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:26.949219 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] PATH_INFO: `/v3/projects/59a60575db5d4454b8404d3ce06dd5af` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:26.950838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a60575db5d4454b8404d3ce06dd5af)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:26.953937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:27.101202 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a60575db5d4454b8404d3ce06dd5af', 'request_id': 'req-b6c62abb-9203-442f-a060-698e07021189'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:27.106825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a60575db5d4454b8404d3ce06dd5af', 'request_id': 'req-b6c62abb-9203-442f-a060-698e07021189'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:27.199029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a60575db5d4454b8404d3ce06dd5af', 'request_id': 'req-b6c62abb-9203-442f-a060-698e07021189'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:27.211510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] The token cache is being invalidate because project 59a60575db5d4454b8404d3ce06dd5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:06:27.211810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:27.212476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:06:27.214033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1295/2591] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:06:26 2022] DELETE /identity/v3/projects/59a60575db5d4454b8404d3ce06dd5af => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:06:36.247821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:37.208136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:38.885005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:38.894676 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c8e62-d715-4e12-a25f-77440800f02d service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:38.947311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:38.947977 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:38.948269 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:38.948492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:38.950389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:39.024908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:39.026835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795ad7-a0ea-4b95-a540-9928508e9f35 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:39.042674 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1297/2592] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:49.033992 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=50 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:50.258778 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:50.260302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:50.273757 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62abb-9203-442f-a060-698e07021189 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:50.277638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:50.278447 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:50.278734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:50.279014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:50.281569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:50.391185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:50.392726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:50.416866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1296/2593] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:51.839814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d4e8-5486-45c7-8d9e-72e34196980c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:51.840359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d4e8-5486-45c7-8d9e-72e34196980c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:51.840359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0d4e8-5486-45c7-8d9e-72e34196980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:51.860654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-bfe0d4e8-5486-45c7-8d9e-72e34196980c None None] MFA Rules not processed for user `564e0f9000a1448ea9ea37a2892e6495`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:51.861884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-bfe0d4e8-5486-45c7-8d9e-72e34196980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:51.902496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:51.903983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:51.916399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b82d-c0ef-49a6-9084-aaeca74942cf service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:51.920427 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:51.921119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1298/2594] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:51.921267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:51.921503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:51.921968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:51.923764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:51.939193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:51.940618 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:51.954123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1297/2595] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:52.755420 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:52.755855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:52.756312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:52.775752 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] MFA Rules not processed for user `f740005b7fc245919a6ddbf61ee9ba85`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:52.782542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:52.838611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1299/2596] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:07:01.947776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=40 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:02.831242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=50 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:06.882038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:06.891736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb27f5-0944-4675-b9e2-d18c91a589bf service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:06.895808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:06.896422 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:06.896705 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:06.897024 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:06.898862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:06.966952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:06.968339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:06.978796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1298/2597] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:08.261008 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:08.272284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8ea6-572e-4637-9e1f-a23f48deaeb4 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:08.276095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:08.276737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:08.277030 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:08.277316 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:08.280388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:08.294256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:08.295803 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:08.306927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1300/2598] 10.222.0.85 () {60 vars in 1396 bytes} [Thu Jul 28 18:07:08 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:16.973752 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=30 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:18.301320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:32.075964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:32.078517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:32.095119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109a37b-b924-458b-bc7a-62d65e87f5d1 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:32.100720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:32.101336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:32.101622 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:32.101904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:32.103748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:32.116692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:32.118341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48559070-1f22-498a-9383-d1abe9a88a1c service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:32.129321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1299/2599] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:39.005768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:39.017045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5b890-4fd4-48a2-88a8-06ca2a914dcf service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:39.022502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:39.023685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:39.024139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:39.024594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:39.029133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:39.056247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:39.058115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:39.067536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1301/2600] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:42.124905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=6 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:44.556299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:44.557154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:44.557154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:44.578998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] MFA Rules not processed for user `aa7941170fe74f9c818f4ab612fca33f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:07:44.580230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:44.645686 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1300/2601] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:07:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:07:44.889511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:44.898609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db41bf-93d0-4de4-b6be-57d0c5ed7eac service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:44.902230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:44.902865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:44.903156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:44.903436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:44.905421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:44.915218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:44.916640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:44.925295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1302/2602] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:45.344494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:45.346042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:45.428449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab244c6a-aa11-44f3-b0ca-f08c2cf8b2a3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:45.433623 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07459c77-374a-4860-b329-578bceddb764 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:45.434987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459c77-374a-4860-b329-578bceddb764 service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:45.434987 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459c77-374a-4860-b329-578bceddb764 service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:45.435200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459c77-374a-4860-b329-578bceddb764 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:45.437070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-07459c77-374a-4860-b329-578bceddb764 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:45.449265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07459c77-374a-4860-b329-578bceddb764 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:45.453751 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07459c77-374a-4860-b329-578bceddb764 service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:45.461172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1301/2603] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:07:51.626767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:51.642304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743218b-c13f-4ec7-8fd2-716050b3ff3a service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:51.648025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:51.648646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:51.648934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:51.649215 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:51.651009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:51.664856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:51.666380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:51.678064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1303/2604] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:51.860228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:51.860614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:51.860790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:51.879934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:07:51.881312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:51.939708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1302/2605] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:07:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:07:52.092309 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:52.094079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:52.171002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af662a-7491-44c7-bdbb-6ac6de45b1ef service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:52.177322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:52.178019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:52.178313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:52.178627 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:52.188220 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:52.190025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:52.198820 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1304/2606] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:07:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:07:55.589684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:55.592668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:55.646169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=100 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:55.660770 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee994b-ee39-4b5c-b4f0-0795bc6baedc None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:55.664694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:55.665361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:55.665645 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:55.665922 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:55.667809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:55.679888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:55.682381 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:55.693084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1303/2607] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:58.024456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:07:58.038414 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d414e-8fa0-471f-b953-df1229bfb4c6 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:07:58.044195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:07:58.045457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:07:58.045918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:07:58.046210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:07:58.048218 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:07:58.061754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:07:58.063464 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:07:58.075064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1305/2608] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:07:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:01.897706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:01.899195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:01.915245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330692-0608-4478-91da-be0c0fac038e service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:01.922744 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:01.923375 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:01.923672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:01.923954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:01.926021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:01.940103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:01.943523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:01.953016 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1304/2609] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:08:02.039352 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:02.041234 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:02.112195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abae6b-3bc3-47d7-8231-72416d2c1c1e service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:02.116976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:02.117622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:02.117911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:02.118195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] PATH_INFO: `/v3/users/78888cc145ba4052be05b1c321de44fd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:02.119502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78888cc145ba4052be05b1c321de44fd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:02.121440 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:02.261091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:02.268198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78888cc145ba4052be05b1c321de44fd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.268880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78888cc145ba4052be05b1c321de44fd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.336372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78888cc145ba4052be05b1c321de44fd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.351856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78888cc145ba4052be05b1c321de44fd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.352719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78888cc145ba4052be05b1c321de44fd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.367119 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1306/2610] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:08:02 2022] DELETE /identity/v3/users/78888cc145ba4052be05b1c321de44fd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:02.629701 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:02.633862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06b24e-a917-454e-afb6-ba5cb5d579ac service placement] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:02.711352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:02.712419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:02.712419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:02.712419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] PATH_INFO: `/v3/projects/39115160ca364b2590af12fccde98256` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:02.712419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39115160ca364b2590af12fccde98256)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:02.715728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:02.820425 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39115160ca364b2590af12fccde98256', 'request_id': 'req-925981d1-b0af-4139-b616-c92c13bd0740'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.826174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39115160ca364b2590af12fccde98256', 'request_id': 'req-925981d1-b0af-4139-b616-c92c13bd0740'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.898125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39115160ca364b2590af12fccde98256', 'request_id': 'req-925981d1-b0af-4139-b616-c92c13bd0740'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.910625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] The token cache is being invalidate because project 39115160ca364b2590af12fccde98256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:08:02.910921 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.911594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:02.913129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1305/2611] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:08:02 2022] DELETE /identity/v3/projects/39115160ca364b2590af12fccde98256 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:02.917303 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:02.918755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:02.934634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4dc9b-4094-4b22-84cd-7b2e638e01e0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:02.993265 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:02.993924 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:02.994255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:02.994723 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] PATH_INFO: `/v3/users/7367c39686814c91aa5e1fd8eff6eb06` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:02.995939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7367c39686814c91aa5e1fd8eff6eb06)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:02.999973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:03.211142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7367c39686814c91aa5e1fd8eff6eb06'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.212208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7367c39686814c91aa5e1fd8eff6eb06'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.315237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7367c39686814c91aa5e1fd8eff6eb06'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.329399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7367c39686814c91aa5e1fd8eff6eb06'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.330299 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7367c39686814c91aa5e1fd8eff6eb06'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.343305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1307/2612] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:08:02 2022] DELETE /identity/v3/users/7367c39686814c91aa5e1fd8eff6eb06 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:03.593124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:03.594999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:03.617318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925981d1-b0af-4139-b616-c92c13bd0740 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:03.699599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:03.700259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:03.700590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:03.700873 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] PATH_INFO: `/v3/projects/54e11078158541c0bafc3b82d2da2d72` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:03.702184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e11078158541c0bafc3b82d2da2d72)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:03.704282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:03.705937 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:03.818011 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e11078158541c0bafc3b82d2da2d72', 'request_id': 'req-f8787ac7-d4ad-4773-9aee-45d533c59faa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.833252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e11078158541c0bafc3b82d2da2d72', 'request_id': 'req-f8787ac7-d4ad-4773-9aee-45d533c59faa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.909524 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e11078158541c0bafc3b82d2da2d72', 'request_id': 'req-f8787ac7-d4ad-4773-9aee-45d533c59faa'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.921635 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] The token cache is being invalidate because project 54e11078158541c0bafc3b82d2da2d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:08:03.921928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.922620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:03.927077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1306/2613] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:08:03 2022] DELETE /identity/v3/projects/54e11078158541c0bafc3b82d2da2d72 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:13.339632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=15,keystone:SELECT=67,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:13.918184 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:25.308862 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:25.310447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:25.435296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ef6c3-5b76-4ffc-a05a-b15960fb1772 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:25.494774 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:25.495406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:25.495691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:25.495971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:25.497812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:25.586427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:25.616398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:25.617885 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:25.641120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1308/2614] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:29.504643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:29.506140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:29.582035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8787ac7-d4ad-4773-9aee-45d533c59faa admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:29.586906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:29.587560 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:29.587862 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:29.588213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] PATH_INFO: `/v3/users/564e0f9000a1448ea9ea37a2892e6495` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:29.589365 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=564e0f9000a1448ea9ea37a2892e6495)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:29.591374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:29.827005 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564e0f9000a1448ea9ea37a2892e6495'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:29.827658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564e0f9000a1448ea9ea37a2892e6495'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:29.828354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564e0f9000a1448ea9ea37a2892e6495'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:29.840750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564e0f9000a1448ea9ea37a2892e6495'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.034911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f841bc35-d021-41ae-81cd-1377588a97b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564e0f9000a1448ea9ea37a2892e6495'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.049034 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1307/2615] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:08:29 2022] DELETE /identity/v3/users/564e0f9000a1448ea9ea37a2892e6495 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:30.310588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:30.320934 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d2f7-6685-44ec-946d-ac444ca3ce3c service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:30.381496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:30.382045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:30.382327 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:30.382631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] PATH_INFO: `/v3/projects/c1c1587cf20d427e8bb03ffae1b1ce33` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:30.383806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c1587cf20d427e8bb03ffae1b1ce33)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:30.385757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:30.493230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c1587cf20d427e8bb03ffae1b1ce33', 'request_id': 'req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.499507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c1587cf20d427e8bb03ffae1b1ce33', 'request_id': 'req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.506205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c1587cf20d427e8bb03ffae1b1ce33', 'request_id': 'req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.593928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] The token cache is being invalidate because project c1c1587cf20d427e8bb03ffae1b1ce33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:08:30.594225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.594902 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ccfa5720-cc1f-4ad8-a13a-f9cbd818b830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:30.597193 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1309/2616] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:08:30 2022] DELETE /identity/v3/projects/c1c1587cf20d427e8bb03ffae1b1ce33 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:30.609348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:30.609885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:30.610071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:30.677576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:30.678769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:30.728099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:30.747559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1308/2617] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:30.756246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:30.756679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:30.757092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:30.780470 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:30.782743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:30.796100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1310/2618] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:30.797656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:30.811555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38a4f-ecee-408b-b07d-651248ac8b6a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:30.818626 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:30.824906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:30.844243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1309/2619] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:08:30.849393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:30.859396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af457228-55e6-46fc-984a-6a229d9f1f00 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:30.865402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:30.866140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:30.866685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:30.867102 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:30.867874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:30.869910 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:30.995680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1311/2620] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:08:30 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:08:31.001226 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:31.011954 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d38ca-3c4f-4af6-b437-3b03f811804e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:31.069847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:31.070513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.070801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.071090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.071758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:31.073682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:31.212593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1310/2621] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:08:31 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:08:31.216875 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:31.227840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2f34-973c-4e66-82dc-4f78b70e0e70 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:31.232873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:31.233483 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.233767 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.234043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.234930 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:31.236520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:31.247237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1312/2622] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:08:31.251402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:31.262673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6adab-8109-41ca-a20d-9c7c371d1dac admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:31.267390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:31.270794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.270794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.270794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.270794 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:31.271200 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0a060-2975-4d93-9da9-e08279523aae admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:31.281731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1311/2623] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:08:31.286173 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:31.296229 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c55e95-941b-4228-bcf5-e49fd103e263 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:31.301523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:31.302178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.302497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.302779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] PATH_INFO: `/v3/projects/f4857a4e9170472da43e01f442099c29/users/f26806f1da1b4e98a867fab87249f9fe/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.311138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:31.319191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4857a4e9170472da43e01f442099c29, user_id=f26806f1da1b4e98a867fab87249f9fe, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:31.321900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:31.423391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1313/2624] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:08:31 2022] PUT /identity/v3/projects/f4857a4e9170472da43e01f442099c29/users/f26806f1da1b4e98a867fab87249f9fe/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:31.429374 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23db5a-9416-4255-812d-9345cebcddfb None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.429667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23db5a-9416-4255-812d-9345cebcddfb None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.429946 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23db5a-9416-4255-812d-9345cebcddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.450440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2d23db5a-9416-4255-812d-9345cebcddfb None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:31.451657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2d23db5a-9416-4255-812d-9345cebcddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:31.516928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1312/2625] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:08:31.527326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:31.529093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:31.589508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1524-3fc8-4431-afa5-4ee8afb006bd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:31.593415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:31.594039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:31.594325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:31.594634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:31.604962 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:31.606689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:31.617446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1314/2626] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:33.493090 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3350-1035-46b0-a331-39f877f2f6b7 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.493806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3350-1035-46b0-a331-39f877f2f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.493806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3350-1035-46b0-a331-39f877f2f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.526679 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4a6f3350-1035-46b0-a331-39f877f2f6b7 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:33.527966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4a6f3350-1035-46b0-a331-39f877f2f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.576210 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:33.591511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1313/2627] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:08:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:33.604182 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.605672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.679952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ae00f-2a13-4dca-9e42-854b1835c398 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.685348 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:33.685971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.686261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.686575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.697270 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:33.698961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:33.722580 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1315/2628] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:08:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:08:38.705963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df5d-8e77-483f-9f7d-6cd3a163a062 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:38.705963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df5d-8e77-483f-9f7d-6cd3a163a062 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:38.705963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60df5d-8e77-483f-9f7d-6cd3a163a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:38.755361 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5f60df5d-8e77-483f-9f7d-6cd3a163a062 None None] MFA Rules not processed for user `f26806f1da1b4e98a867fab87249f9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:38.759934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5f60df5d-8e77-483f-9f7d-6cd3a163a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:38.870430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1314/2629] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:08:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:38.878438 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:38.880929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:39.006428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd380-ad61-4d3e-8771-b647435e18a5 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:39.020159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:39.020953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:39.020953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:39.021224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:39.031466 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:39.034290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aad8f-b090-46c0-927d-f30c5bf54d40 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:39.049262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1316/2630] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:43.060318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3df2d-0448-4ffc-ae76-0ba03354fc70 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:43.060799 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3df2d-0448-4ffc-ae76-0ba03354fc70 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:43.060907 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3df2d-0448-4ffc-ae76-0ba03354fc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:43.083180 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e3e3df2d-0448-4ffc-ae76-0ba03354fc70 None None] MFA Rules not processed for user `f26806f1da1b4e98a867fab87249f9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:43.092448 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e3e3df2d-0448-4ffc-ae76-0ba03354fc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:43.111765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1315/2631] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:08:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:49.043412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:53.081144 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=39 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:59.190649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.191776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.191776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.193142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1317/2632] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:08:59.200094 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749b97-8115-4eb7-8334-f34820c5fb3b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.200707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749b97-8115-4eb7-8334-f34820c5fb3b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.200707 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9749b97-8115-4eb7-8334-f34820c5fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.223001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-d9749b97-8115-4eb7-8334-f34820c5fb3b None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:59.224349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d9749b97-8115-4eb7-8334-f34820c5fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:59.234035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1316/2633] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:08:59.246019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:59.250036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:59.313993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f3d5e-d4a0-47a8-8f45-f9123d06a986 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:59.319428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:59.319846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.320128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.320402 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.332693 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:59.334198 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f2410-8d8e-4683-b0c5-3d744e67ca6b service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:59.347613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1318/2634] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:59.630301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f8af-f829-4dad-9bcb-a7c0497a6c9a None None] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.630604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f8af-f829-4dad-9bcb-a7c0497a6c9a None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.630979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f8af-f829-4dad-9bcb-a7c0497a6c9a None None] PATH_INFO: `/` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.632554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1317/2635] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:08:59.637823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe36d9-5c46-440c-af82-40cd5288137f None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.638301 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe36d9-5c46-440c-af82-40cd5288137f None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.640067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe36d9-5c46-440c-af82-40cd5288137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.662762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a4fe36d9-5c46-440c-af82-40cd5288137f None None] MFA Rules not processed for user `14ebf672346841c8addaa849fc11a052`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:59.664462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a4fe36d9-5c46-440c-af82-40cd5288137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:59.671584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1319/2636] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:08:59.685103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64f8af-f829-4dad-9bcb-a7c0497a6c9a None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:59.698412 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64f8af-f829-4dad-9bcb-a7c0497a6c9a None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:59.702383 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:59.703003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.703290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.703614 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.712856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:59.714368 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb805f1c-d7b5-474f-b720-1631df1e3dee service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:59.724864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1318/2637] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:59.815748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705feb8-ecf9-4662-9d9f-20e384e83aa3 None None] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.816039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705feb8-ecf9-4662-9d9f-20e384e83aa3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.816319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705feb8-ecf9-4662-9d9f-20e384e83aa3 None None] PATH_INFO: `/` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.817743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1320/2638] 10.222.0.85 () {54 vars in 921 bytes} [Thu Jul 28 18:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:08:59.823821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514321f-678c-42f9-925a-a66c12244c7d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.824199 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514321f-678c-42f9-925a-a66c12244c7d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.824600 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514321f-678c-42f9-925a-a66c12244c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.845406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5514321f-678c-42f9-925a-a66c12244c7d None None] MFA Rules not processed for user `1e396c6de22e485bb62fcecfd7f49243`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:59.846809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5514321f-678c-42f9-925a-a66c12244c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:59.857055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1319/2639] 10.222.0.85 () {58 vars in 1018 bytes} [Thu Jul 28 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:59.869148 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705feb8-ecf9-4662-9d9f-20e384e83aa3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:59.881320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705feb8-ecf9-4662-9d9f-20e384e83aa3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:59.885021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:59.885636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:59.885927 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:59.886205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:59.898543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:59.900001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:59.912450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1321/2640] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:09:04.301007 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:04.301549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:04.301637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:04.325845 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:09:04.327132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:04.335449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1320/2641] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:09:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:09:04.362825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:09:04.378093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b329-7124-4aa1-a102-d34d6e0d67d2 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:09:04.384276 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:09:04.388129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:04.388129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:04.388129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:04.396201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:09:04.399111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:09:04.408846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1322/2642] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:09:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:09:05.737081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:09:05.747721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:05.761279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c076ae9-f78a-4d29-9fe6-be3dd27b9106 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:09:05.767392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:09:05.768384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:05.768663 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:05.769170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] PATH_INFO: `/v3/users/aa7941170fe74f9c818f4ab612fca33f` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:05.770760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7941170fe74f9c818f4ab612fca33f)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:09:05.773513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:09:05.999708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7941170fe74f9c818f4ab612fca33f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.000385 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7941170fe74f9c818f4ab612fca33f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.001530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7941170fe74f9c818f4ab612fca33f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.023081 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7941170fe74f9c818f4ab612fca33f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.130238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f977ecc0-926c-48d2-9620-81bf5cc17baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7941170fe74f9c818f4ab612fca33f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.145850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1321/2643] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:09:05 2022] DELETE /identity/v3/users/aa7941170fe74f9c818f4ab612fca33f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:09:06.447057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:09:06.460507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417d519-28ea-49d8-875d-b5feb29372d4 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:09:06.555608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:09:06.556886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:06.557468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:06.557981 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] PATH_INFO: `/v3/projects/613c6d4f37814a209077ac2c19506674` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:06.559596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613c6d4f37814a209077ac2c19506674)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:09:06.562117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:09:06.692156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613c6d4f37814a209077ac2c19506674', 'request_id': 'req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.699293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613c6d4f37814a209077ac2c19506674', 'request_id': 'req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.706773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613c6d4f37814a209077ac2c19506674', 'request_id': 'req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.915613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:09:06.923010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] The token cache is being invalidate because project 613c6d4f37814a209077ac2c19506674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:09:06.924297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.924493 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:09:06.926069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1323/2644] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:09:06 2022] DELETE /identity/v3/projects/613c6d4f37814a209077ac2c19506674 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:09:16.142004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:09:19.604278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf96846-38cf-44af-a0c7-d6571aa6aa83 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:19.605580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf96846-38cf-44af-a0c7-d6571aa6aa83 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:19.605580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf96846-38cf-44af-a0c7-d6571aa6aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:19.629163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-4cf96846-38cf-44af-a0c7-d6571aa6aa83 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:09:19.630814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4cf96846-38cf-44af-a0c7-d6571aa6aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:19.697230 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1322/2645] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:09:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:09:19.710773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:09:19.711687 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:19.789179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfed40-ff71-47c9-9504-c88cf7fdf9d2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:09:19.794364 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:09:19.794993 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:19.795348 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:19.795558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:19.810416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:09:19.812117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:09:19.821647 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1324/2646] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:09:20.193414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:20.193743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:20.194023 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:20.214333 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] MFA Rules not processed for user `73edfd05da254457b6ad9c9c9f662e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:09:20.215967 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:20.279643 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1323/2647] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:09:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:09:20.645150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:09:20.646984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:20.707406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958311e3-fe6b-40f7-8c15-a4446a2d235e service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:09:20.711135 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:09:20.711766 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:09:20.712047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:09:20.712325 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:09:20.714104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:09:20.724923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:09:20.726481 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:09:20.736850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1325/2648] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:09:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:09:30.270768 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=50 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:09:30.732545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=58 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:06.507161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:06.516755 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363e49e-d643-47ee-8012-30bb5f0c65ed None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:06.520790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:06.521392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:06.521672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:06.521945 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:06.523696 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:06.585671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:06.587145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:06.596876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1324/2649] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:09.941678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:09.943542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:10.036669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18647293-4ecb-423d-a2c9-4f1e3aff0696 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:10.041760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:10.045029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:10.045029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:10.045029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:10.045029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:10.057614 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:10.059233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:10.081040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1326/2650] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:10.111697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:10.113190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:10.126038 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a129f-a838-4d09-9a3e-b3aeb31871e0 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:10.130831 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:10.131475 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:10.131781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:10.132063 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] PATH_INFO: `/v3/users/f740005b7fc245919a6ddbf61ee9ba85` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:10.133235 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f740005b7fc245919a6ddbf61ee9ba85)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:10.135248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:10.335958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f740005b7fc245919a6ddbf61ee9ba85'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.336642 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f740005b7fc245919a6ddbf61ee9ba85'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.337382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f740005b7fc245919a6ddbf61ee9ba85'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.352316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f740005b7fc245919a6ddbf61ee9ba85'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.418893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-f6b641a2-d6e1-4cdf-b383-9806e7730434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f740005b7fc245919a6ddbf61ee9ba85'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.431786 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1325/2651] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:10:10 2022] DELETE /identity/v3/users/f740005b7fc245919a6ddbf61ee9ba85 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:10:10.695702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:10.734202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b7f2-d6a3-4d78-b299-ee666590aefa service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:10.792702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:10.793346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:10.793632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:10.793911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] PATH_INFO: `/v3/projects/2732f627abb44936bedd3293898b24f3` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:10.795200 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2732f627abb44936bedd3293898b24f3)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:10.797115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:10.899813 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2732f627abb44936bedd3293898b24f3', 'request_id': 'req-6b2e8bc5-7868-483c-b044-1c07b129464b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.906432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2732f627abb44936bedd3293898b24f3', 'request_id': 'req-6b2e8bc5-7868-483c-b044-1c07b129464b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.912654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2732f627abb44936bedd3293898b24f3', 'request_id': 'req-6b2e8bc5-7868-483c-b044-1c07b129464b'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.969212 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:10.974163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] The token cache is being invalidate because project 2732f627abb44936bedd3293898b24f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:10:10.974497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.975192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6b2e8bc5-7868-483c-b044-1c07b129464b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:10:10.978529 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1327/2652] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:10:10 2022] DELETE /identity/v3/projects/2732f627abb44936bedd3293898b24f3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:10:10.981966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:10.982282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:10.982648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.002159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:11.003399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:11.068276 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1326/2653] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:11.070883 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.071253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.071617 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.092167 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:11.093380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:11.102615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1328/2654] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:10:11.106981 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.119773 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a78297-b2e0-45b3-a804-5a1f733f9c40 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.122884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.123520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.123810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.124328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.125172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:10:11.125483 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.127171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.128857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:11.140628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1327/2655] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:10:11.145637 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.155395 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5044132-207b-4cf7-b06d-0da11a17fd33 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.161261 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.161956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.162504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.162866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.163638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.165643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.255278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1329/2656] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:10:11 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:10:11.259500 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.270784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38bc92-6be0-4ffa-8e6a-199fd072fad6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.328156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.328790 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.329061 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.330029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.330727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.332633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.419748 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1328/2657] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:10:11 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:10:11.424050 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.434495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf8cb0-0d40-405b-872d-cdbf9b0753d3 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.440052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.442680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.442680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.442680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.442680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.443523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.453408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1330/2658] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:10:11.457980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.467979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c064ddf-c3f0-4c9a-af63-a143aaa6f5b2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.472878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.473482 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.473767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.474044 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.474847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.476450 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0edbec-8a9d-4b23-8464-5dc9b06fb0ee admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.488342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1329/2659] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:10:11.492498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.503306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fceb60-b1a4-4cd0-82ad-ba38adda1f63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.508023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.508708 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.508999 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.509285 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] PATH_INFO: `/v3/projects/b8daf9e3c7be417c82e7cbf8624d1d5d/users/de20d8b31db44abf91a482d81b898755/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.525717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8daf9e3c7be417c82e7cbf8624d1d5d, user_id=de20d8b31db44abf91a482d81b898755, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.528476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.596000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1331/2660] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:10:11 2022] PUT /identity/v3/projects/b8daf9e3c7be417c82e7cbf8624d1d5d/users/de20d8b31db44abf91a482d81b898755/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:10:11.605943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67a51-959b-4894-9389-b2368963e04e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.606238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67a51-959b-4894-9389-b2368963e04e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.606555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d67a51-959b-4894-9389-b2368963e04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.626717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b9d67a51-959b-4894-9389-b2368963e04e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:11.628128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b9d67a51-959b-4894-9389-b2368963e04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:11.686836 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1330/2661] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:11.697029 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:11.698785 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:11.760945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4e249-4b91-4993-a36b-7965dcc826c8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:11.764690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:11.765312 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:11.765597 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:11.765877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:11.776139 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:11.777625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:11.787796 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1332/2662] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:17.128169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf907-cc5f-4864-8c86-6ce4bc505b92 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:17.128702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf907-cc5f-4864-8c86-6ce4bc505b92 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:17.128815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bf907-cc5f-4864-8c86-6ce4bc505b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:17.160913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-623bf907-cc5f-4864-8c86-6ce4bc505b92 None None] MFA Rules not processed for user `de20d8b31db44abf91a482d81b898755`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:17.162437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-623bf907-cc5f-4864-8c86-6ce4bc505b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:17.277221 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1331/2663] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:10:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:17.290007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:17.291782 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:17.434459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefd862-10f6-458b-8dd5-0cdbbe3b09ee service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:17.441486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:17.442140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:17.442446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:17.442725 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:17.451638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:17.453255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:17.463861 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:17.464273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1333/2664] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:19.844001 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:19.844468 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:19.845417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:19.853397 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:19.867843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] MFA Rules not processed for user `de20d8b31db44abf91a482d81b898755`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:19.869395 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:19.876762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1332/2665] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:10:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:22.224791 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:22.227343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:22.296206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ab48-88bf-45f2-9953-5b51a28667e7 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:22.301287 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:22.301908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.302195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.302511 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:22.304255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:22.315748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:22.317328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:22.327162 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1334/2666] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:29.966750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:32.323068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=33 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:11:19.015139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:19.016833 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:19.017178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:19.018784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:19.034853 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091b6af-985c-485f-9cc1-78010af46ba0 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:19.040453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:19.041073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:19.041971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:19.042304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:19.044052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:19.109895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:19.111535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:19.116335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5148f19-2418-4103-8355-d68b5c561fe5 service placement] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:19.122612 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1333/2667] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:11:19.127304 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:19.128245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:19.128534 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:19.128915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:19.140190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:19.141696 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2dc88-3e32-4b90-99bc-696d84e5ac1e service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:19.152037 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1335/2668] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:11:21.785312 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:21.788032 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:21.801848 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda13c-4386-4cc5-838f-a5cd59ebfa76 service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:21.805874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:21.806552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:21.807360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:21.807637 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:21.810314 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:21.827248 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:21.828720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:21.850277 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1334/2669] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:11:24.591485 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:24.592140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:24.592140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:24.621994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] MFA Rules not processed for user `de20d8b31db44abf91a482d81b898755`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:11:24.623423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:24.635916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1336/2670] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:11:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:11:28.901517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:28.903845 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:28.924242 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87824-5ea8-44d8-8689-1550456b077d service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:28.929464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:28.930133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:28.930472 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:28.930758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:28.932692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:29.019711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:29.021289 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:29.036432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1335/2671] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:11:31.133941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:31.135497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:31.150211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206598a-9f4b-48de-9f8b-9af039af1bf0 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:31.155142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:31.155805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:31.159523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:31.159523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] PATH_INFO: `/v3/users/186ea84ee37d48c186190438adce39b9` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:31.159523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=186ea84ee37d48c186190438adce39b9)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:31.159523 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:31.350648 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186ea84ee37d48c186190438adce39b9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.351383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186ea84ee37d48c186190438adce39b9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.416566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186ea84ee37d48c186190438adce39b9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.430067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186ea84ee37d48c186190438adce39b9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.430837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186ea84ee37d48c186190438adce39b9'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.444399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1337/2672] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:11:31 2022] DELETE /identity/v3/users/186ea84ee37d48c186190438adce39b9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:11:31.721729 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:31.736299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c45c5-bde4-46f9-9e71-a29e875b562d service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:31.824683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:31.825350 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:31.825652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:31.825925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] PATH_INFO: `/v3/projects/efa7968cb99d4470b48f8ff2fd8bad7a` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:31.827124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa7968cb99d4470b48f8ff2fd8bad7a)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:31.829046 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:31.966128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:11:31.984533 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa7968cb99d4470b48f8ff2fd8bad7a', 'request_id': 'req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:31.990473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa7968cb99d4470b48f8ff2fd8bad7a', 'request_id': 'req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.072924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa7968cb99d4470b48f8ff2fd8bad7a', 'request_id': 'req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.085444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] The token cache is being invalidate because project efa7968cb99d4470b48f8ff2fd8bad7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:11:32.085756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.087347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.088163 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1336/2673] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:11:31 2022] DELETE /identity/v3/projects/efa7968cb99d4470b48f8ff2fd8bad7a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:11:32.092538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:32.094138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:32.109001 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918798-3bcf-4190-b6d4-5b7b2a62a837 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:32.157158 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:11:32.169699 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:32.170621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:32.170911 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:32.171188 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] PATH_INFO: `/v3/users/10550bec9e91406880734d9543cfe8bd` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:32.172375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10550bec9e91406880734d9543cfe8bd)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:32.174422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:32.397328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10550bec9e91406880734d9543cfe8bd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.398082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10550bec9e91406880734d9543cfe8bd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.484169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10550bec9e91406880734d9543cfe8bd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.498267 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10550bec9e91406880734d9543cfe8bd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.499121 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a2fae682-c74c-46ef-b9ee-f227a246f453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10550bec9e91406880734d9543cfe8bd'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:32.512804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1338/2674] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:11:32 2022] DELETE /identity/v3/users/10550bec9e91406880734d9543cfe8bd => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:11:32.815050 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:32.816608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:32.830114 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c28bf-d806-43f8-88e1-4dc97a6fc9ad admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:32.933721 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:32.934520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:32.934866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:32.935257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] PATH_INFO: `/v3/projects/a0a527edebc54195bb89297144e5cae4` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:32.936510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a527edebc54195bb89297144e5cae4)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:32.938466 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:33.167194 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a527edebc54195bb89297144e5cae4', 'request_id': 'req-bb43594a-1a13-4700-b7f7-6a734127bbc1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:33.175029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a527edebc54195bb89297144e5cae4', 'request_id': 'req-bb43594a-1a13-4700-b7f7-6a734127bbc1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:33.312606 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a527edebc54195bb89297144e5cae4', 'request_id': 'req-bb43594a-1a13-4700-b7f7-6a734127bbc1'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:33.326169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] The token cache is being invalidate because project a0a527edebc54195bb89297144e5cae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:11:33.326493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:33.327152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:11:33.328664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1337/2675] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:11:32 2022] DELETE /identity/v3/projects/a0a527edebc54195bb89297144e5cae4 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:11:38.930893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:38.931454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:38.931454 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:38.951836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] MFA Rules not processed for user `73edfd05da254457b6ad9c9c9f662e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:11:38.952278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:39.011874 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1339/2676] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:11:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:11:43.322540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:11:49.004208 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:11:50.437650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:50.439185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:50.501173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43594a-1a13-4700-b7f7-6a734127bbc1 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:50.505185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:50.505808 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:50.506115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:50.506432 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:50.508340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:50.581304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:50.582793 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721b5ca-d239-4538-94fe-af8199707b16 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:50.594823 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1338/2677] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:11:52.960020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:11:52.961497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:52.974769 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382d67-3e96-48bb-aa06-c302a2b83a83 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:11:52.981429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:11:52.981429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:11:52.981429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:11:52.981429 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:11:52.982643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:11:52.996093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:11:52.997542 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:11:53.008623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1340/2678] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:00.131458 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527784-98c6-4775-9f4f-c51c1f24b2a1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:00.134093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527784-98c6-4775-9f4f-c51c1f24b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:00.134093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527784-98c6-4775-9f4f-c51c1f24b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:00.152629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2d527784-98c6-4775-9f4f-c51c1f24b2a1 None None] MFA Rules not processed for user `f26806f1da1b4e98a867fab87249f9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:00.154767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2d527784-98c6-4775-9f4f-c51c1f24b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:00.229197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1339/2679] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:12:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:00.506815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:00.515707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30706d27-b04a-434b-bd3d-4d69c245209b service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:00.519601 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:00.523291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:00.523291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:00.523291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:00.523291 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:00.591318 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:00.592778 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:00.603243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1341/2680] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:10.218821 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=79 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:10.598538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=36 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:35.095783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.096356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.096356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.116580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] MFA Rules not processed for user `de20d8b31db44abf91a482d81b898755`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:35.117810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:35.174464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1340/2681] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:38.950011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:38.959524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353921d0-34f8-41e1-9c78-37ff086e0f9a service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:38.963201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:38.963808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:38.964095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:38.964371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:38.966192 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:38.976784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:38.978432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:38.986987 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1342/2682] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:39.847646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:39.855249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:39.874969 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe29dd1-3d43-46d3-a2ee-7ee510c414f1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:39.881652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:39.882298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:39.882651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:39.882936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:39.884903 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:39.906813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:39.906813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:39.913605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1341/2683] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:48.983331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:49.909225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=31 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:31.010434 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:31.021506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7d923-c38f-414a-abe6-7a12cbf8bcb1 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:31.025282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:31.025939 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:31.026211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:31.026525 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:31.029228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:31.094660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:31.096380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:31.106704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1343/2684] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:32.383809 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:32.385820 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:32.402883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82155942-ddb2-4289-99a6-9378cd702b7a service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:32.410584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:32.411252 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:32.411535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:32.411814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:32.413632 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:32.428402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:32.429989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:32.442939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1342/2685] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:32.488733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:32.488733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:32.501408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403e081-b21c-4f84-b9b0-baf4051e9c2e service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:32.506997 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:32.507764 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:32.508228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:32.508702 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] PATH_INFO: `/v3/users/f26806f1da1b4e98a867fab87249f9fe` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:32.510110 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26806f1da1b4e98a867fab87249f9fe)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:32.513093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:32.711900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26806f1da1b4e98a867fab87249f9fe'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:32.712920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26806f1da1b4e98a867fab87249f9fe'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:32.822375 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26806f1da1b4e98a867fab87249f9fe'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:32.853257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26806f1da1b4e98a867fab87249f9fe'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:32.854223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-f7af0599-d8f5-41eb-b97e-95dc4e2b0673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26806f1da1b4e98a867fab87249f9fe'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:32.877557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1344/2686] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:32 2022] DELETE /identity/v3/users/f26806f1da1b4e98a867fab87249f9fe => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:33.189984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:33.203308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8b3e5-eb54-4159-83d2-56cf11fd684f service nova] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:33.268455 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:33.272775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.272775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.272775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] PATH_INFO: `/v3/projects/f4857a4e9170472da43e01f442099c29` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.272775 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4857a4e9170472da43e01f442099c29)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:33.274212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:33.408346 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4857a4e9170472da43e01f442099c29', 'request_id': 'req-74037ff5-1975-4b6c-b417-573653752396'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:33.415279 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4857a4e9170472da43e01f442099c29', 'request_id': 'req-74037ff5-1975-4b6c-b417-573653752396'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:33.544085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4857a4e9170472da43e01f442099c29', 'request_id': 'req-74037ff5-1975-4b6c-b417-573653752396'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:33.556948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] The token cache is being invalidate because project f4857a4e9170472da43e01f442099c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:33.557373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:33.558042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-74037ff5-1975-4b6c-b417-573653752396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:33.560052 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1343/2687] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:33 2022] DELETE /identity/v3/projects/f4857a4e9170472da43e01f442099c29 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:33.565189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6e32-cad0-44e5-a432-319be021738a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.565747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6e32-cad0-44e5-a432-319be021738a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.565843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6e32-cad0-44e5-a432-319be021738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.590718 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-845e6e32-cad0-44e5-a432-319be021738a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:33.592064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-845e6e32-cad0-44e5-a432-319be021738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:33.658498 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1345/2688] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:33.666841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.671161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.671161 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.688074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:33.689452 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:33.698003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1344/2689] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:33.703068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6e32-cad0-44e5-a432-319be021738a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:33.713995 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6e32-cad0-44e5-a432-319be021738a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:33.720514 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:33.728754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:33.729328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:33.737384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1346/2690] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:33.742049 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:33.751959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4c2c-4536-4eb8-9a6b-f8e4896d3195 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:33.758504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:33.759124 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.759405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.759680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.760386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:33.762874 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:33.860062 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1345/2691] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:33 2022] POST /identity/v3/projects => generated 355 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:33.864251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:33.875065 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06610ac4-4faf-4bc8-aa62-0da1b7adcefd admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:33.931876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:33.932502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:33.932789 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:33.933068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:33.934733 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:33.936753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.052042 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1347/2692] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:33 2022] POST /identity/v3/users => generated 278 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:34.058386 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.070489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4288e4-9d55-4158-862d-9dc00f5fc107 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.075994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.076887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.077273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.077766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.078640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.080384 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.093566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1346/2693] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:34.097773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.109818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02c39-bfdf-4a4b-a3a7-598789c13522 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.115072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.116150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.116150 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.116321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.116956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.118815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99f3c7-7384-4ece-9351-06513af1f52d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.130380 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1348/2694] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:34.134648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.145236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2179c-bdad-4e4e-9329-f3d8f8109f1d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.150058 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.150979 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.151267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.151547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] PATH_INFO: `/v3/projects/774ede3299714976905bfba64772d234/users/0d4321437fbb4388a6ae53429d51d360/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.170964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774ede3299714976905bfba64772d234, user_id=0d4321437fbb4388a6ae53429d51d360, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.173765 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.239530 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1347/2695] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:34 2022] PUT /identity/v3/projects/774ede3299714976905bfba64772d234/users/0d4321437fbb4388a6ae53429d51d360/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:34.252584 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.252877 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.253164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.281795 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] MFA Rules not processed for user `0d4321437fbb4388a6ae53429d51d360`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:34.283038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:34.351007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1349/2696] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:34 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:34.356030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.369610 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:34.419582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242dce4-693c-4079-87f7-c8da4f73e4b8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.424282 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.424895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.425179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.425457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.426131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.428310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.545651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1348/2697] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:34 2022] POST /identity/v3/projects => generated 357 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:34.549618 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.562070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540bb14-1aff-4235-ae08-f00c8001e1fc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.627695 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.631569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.631569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.631569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.631569 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.632047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.641914 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:34.724385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1350/2698] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:34 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:34.725891 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.738119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff2d3b-205c-4815-9ec7-fa18e20caf4f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.743715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.744418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.744769 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.745116 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.746037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.747750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.759234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1349/2699] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:34.763321 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.773571 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3bc88-0a25-4088-8018-a4bf5e36b56c admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.779528 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.780213 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.780506 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.780793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] PATH_INFO: `/v3/projects/687f55362dee4a87848d02dde47b862f/users/ffdc11e1792d4a63a398009686de7f6f/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.800753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=687f55362dee4a87848d02dde47b862f, user_id=ffdc11e1792d4a63a398009686de7f6f, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.803520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.888145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1351/2700] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:34 2022] PUT /identity/v3/projects/687f55362dee4a87848d02dde47b862f/users/ffdc11e1792d4a63a398009686de7f6f/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:34.892306 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:34.903735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151ce-5dde-4ac3-9c7a-bee60085180c admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:34.970072 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:34.970743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:34.971030 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:34.971310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:34.972028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:34.973750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4383a1-1af5-46b4-8c53-8bebe6d5f8f5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:34.986340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1350/2701] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:34.990873 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:35.004561 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79714e4a-cef7-4fc1-adca-22fd1d3d9fce admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:35.011383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:35.013596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:35.013596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:35.013596 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] PATH_INFO: `/v3/projects/687f55362dee4a87848d02dde47b862f/users/ffdc11e1792d4a63a398009686de7f6f/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:35.014922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687f55362dee4a87848d02dde47b862f, user_id=ffdc11e1792d4a63a398009686de7f6f, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:35.017610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:35.029818 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1352/2702] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:34 2022] PUT /identity/v3/projects/687f55362dee4a87848d02dde47b862f/users/ffdc11e1792d4a63a398009686de7f6f/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:35.040281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:35.040582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:35.040866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:35.072619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] MFA Rules not processed for user `ffdc11e1792d4a63a398009686de7f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:35.074519 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:35.156646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1351/2703] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:35 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:35.170126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:35.173011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:35.250573 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5931cdf-66fd-417c-ab88-6ae49b419ad2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:35.255897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:35.256537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:35.256830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:35.257115 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:35.269490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:35.271544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420f38c-eae8-4037-ab63-91ccb9aaae8b service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:35.299520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1353/2704] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:35 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:35.501585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:35.516718 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33100-38f0-4089-9009-407ec6bd9606 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:35.527259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:35.528274 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:35.528576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:35.528911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:35.536639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:35.611955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:35.611955 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:35.619915 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1352/2705] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:35 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:45.277550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:45.771347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=24 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:48.781165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:48.781784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:48.781784 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:48.800315 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] MFA Rules not processed for user `de20d8b31db44abf91a482d81b898755`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:48.801522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:48.861109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1354/2706] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:49.195120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:49.196588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.257731 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2cca0-30c3-48c0-ade8-4c32c2f71f54 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:49.261586 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:49.262264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:49.270780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:49.270780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:49.270780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.274566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:49.276517 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:49.285164 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1353/2707] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:49.593600 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:49.595415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.609343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3062f-b89e-41c8-8114-3a1a586a6ad3 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:49.613103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:49.613740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:49.614023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:49.614305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:49.616290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.683320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:49.684777 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:49.694951 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1355/2708] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:49.778067 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:49.780882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.876417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f0deb-1350-4938-8b04-928497297f8b service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:49.881925 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:49.882590 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:49.882877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:49.883156 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:49.884952 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:49.897854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:49.900631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:49.911788 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1354/2709] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:53.853719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.872611 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1aa1d-8ac5-4084-b39b-3ddad6e6a967 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.878546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.879462 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.879906 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.880340 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.883816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:53.900822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.903330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f98cc-21ba-42be-bbbe-c72f98346fd3 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.915103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1356/2710] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:55.645454 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:55.657028 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f5e86-eb7f-4b2f-b6a1-951f5b01e17e service placement] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:55.662473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:55.663127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:55.663417 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:55.663838 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] PATH_INFO: `/v3/users/0d4321437fbb4388a6ae53429d51d360` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:55.665026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4321437fbb4388a6ae53429d51d360)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:55.667534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:55.933670 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4321437fbb4388a6ae53429d51d360'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:55.934370 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4321437fbb4388a6ae53429d51d360'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:55.935220 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4321437fbb4388a6ae53429d51d360'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:55.949585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4321437fbb4388a6ae53429d51d360'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.034123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4321437fbb4388a6ae53429d51d360'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.058639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1355/2711] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:55 2022] DELETE /identity/v3/users/0d4321437fbb4388a6ae53429d51d360 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:56.059747 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.060240 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.060339 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:56.083751 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:56.085305 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.158064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1357/2712] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:56.167814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:56.169251 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.177398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:56.232668 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da51b3e-a71d-45a2-b5ff-acc91b0d9c12 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:56.236473 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:56.237111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.237398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.237680 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:56.248019 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:56.248888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:56.262249 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1356/2713] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:56.704904 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:56.716275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.749092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce891e1-0ccc-4d51-a420-9372697d1523 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:56.763202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:56.764271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.764271 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.764518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] PATH_INFO: `/v3/projects/774ede3299714976905bfba64772d234` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:56.765744 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774ede3299714976905bfba64772d234)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:56.767719 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:56.801247 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:56.918707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774ede3299714976905bfba64772d234', 'request_id': 'req-b70fe965-afda-4327-8b35-bd8ef591b997'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.925677 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774ede3299714976905bfba64772d234', 'request_id': 'req-b70fe965-afda-4327-8b35-bd8ef591b997'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.935471 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774ede3299714976905bfba64772d234', 'request_id': 'req-b70fe965-afda-4327-8b35-bd8ef591b997'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.016986 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] The token cache is being invalidate because project 774ede3299714976905bfba64772d234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:57.017284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.017957 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.019566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1358/2714] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:56 2022] DELETE /identity/v3/projects/774ede3299714976905bfba64772d234 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:57.023647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:57.037742 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f970402-dd36-4516-8383-fa2ad37d9709 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:57.102704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:57.103363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:57.103650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:57.103963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] PATH_INFO: `/v3/users/ffdc11e1792d4a63a398009686de7f6f` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:57.105669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdc11e1792d4a63a398009686de7f6f)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:57.107625 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:57.328938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdc11e1792d4a63a398009686de7f6f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.329659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdc11e1792d4a63a398009686de7f6f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.330275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdc11e1792d4a63a398009686de7f6f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.343151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdc11e1792d4a63a398009686de7f6f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.428366 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-d35e10ff-0d20-40d4-81c2-ae7157698b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdc11e1792d4a63a398009686de7f6f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.445569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1357/2715] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:57 2022] DELETE /identity/v3/users/ffdc11e1792d4a63a398009686de7f6f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:57.943455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:57.953851 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:57.966013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fe965-afda-4327-8b35-bd8ef591b997 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.055331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.056052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.056345 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.056636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] PATH_INFO: `/v3/projects/687f55362dee4a87848d02dde47b862f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.058533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687f55362dee4a87848d02dde47b862f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.068445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.189682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687f55362dee4a87848d02dde47b862f', 'request_id': 'req-d47d7077-10f8-497e-bc33-11e3c0ffbad3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.189682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687f55362dee4a87848d02dde47b862f', 'request_id': 'req-d47d7077-10f8-497e-bc33-11e3c0ffbad3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.199496 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687f55362dee4a87848d02dde47b862f', 'request_id': 'req-d47d7077-10f8-497e-bc33-11e3c0ffbad3'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.357335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] The token cache is being invalidate because project 687f55362dee4a87848d02dde47b862f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:58.357670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.358418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-d47d7077-10f8-497e-bc33-11e3c0ffbad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.360494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1359/2716] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:57 2022] DELETE /identity/v3/projects/687f55362dee4a87848d02dde47b862f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:58.367697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.367697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.367697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.395074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:58.396557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:58.452651 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:58.500154 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1358/2717] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:58.508295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.510971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.510971 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.530557 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:58.531876 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:58.540199 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1360/2718] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:58.544762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:58.555553 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e573e4-0acd-43ea-a6e5-fcc0564db450 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.561045 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.568204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.579126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1359/2719] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:58.583405 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:58.594204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244b220-25df-430f-b830-b6361e7011e9 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.603216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.603929 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.604207 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.604488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.605216 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.607187 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.774382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1361/2720] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:58 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:58.777520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:58.791201 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbf2a1-063f-4937-8335-c4f2ce4afcfc admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.853835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.854509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.854795 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.855683 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.856151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.858139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.960065 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1360/2721] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:58 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:58.964384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:58.975707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc43e-d347-4a9e-a80b-13e649c9c6cc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.980479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.981100 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.981384 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.981664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.982389 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.983994 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.989455 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:58.995054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1362/2722] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:58.999093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.009642 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.010985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e39e9-9797-4e39-b70a-1b73cdf302d2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.012154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.016026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.016654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.016942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.017222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.017950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.019604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.028654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8cc3-b769-473e-b0f7-eccd8483e307 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.033181 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1361/2723] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:59.033533 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.034282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.034612 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.034922 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] PATH_INFO: `/v3/users/73edfd05da254457b6ad9c9c9f662e4c` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.036577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73edfd05da254457b6ad9c9c9f662e4c)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.038495 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.038634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.049514 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687385-5586-4778-ba6a-acafc82b9fc1 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.054595 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.055265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.055554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.055911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] PATH_INFO: `/v3/projects/986f7e3ba4ee4cb0904579f1d8342f36/users/0b6e1e61fc6845d0afb742152e55774d/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.074115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986f7e3ba4ee4cb0904579f1d8342f36, user_id=0b6e1e61fc6845d0afb742152e55774d, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.077938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78ebba-22d8-4545-8549-dd25d0947567 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.239404 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1362/2724] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:59 2022] PUT /identity/v3/projects/986f7e3ba4ee4cb0904579f1d8342f36/users/0b6e1e61fc6845d0afb742152e55774d/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:59.309334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.309763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.313868 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.336712 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73edfd05da254457b6ad9c9c9f662e4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.337386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73edfd05da254457b6ad9c9c9f662e4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.343345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] MFA Rules not processed for user `0b6e1e61fc6845d0afb742152e55774d`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:59.349285 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.410401 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:59.416338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1363/2725] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:59.420884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.428023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73edfd05da254457b6ad9c9c9f662e4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.443918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73edfd05da254457b6ad9c9c9f662e4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.444740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73edfd05da254457b6ad9c9c9f662e4c'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.458448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1363/2726] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:59 2022] DELETE /identity/v3/users/73edfd05da254457b6ad9c9c9f662e4c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:59.489096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f4a6-737d-4f2d-bcb3-d8c08dec0ebe None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.561281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.562103 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.562567 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.562985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.563857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.567304 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.715361 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.716900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.731073 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1364/2727] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/projects => generated 387 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:59.732173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.733983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.808990 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2f859-32b5-4322-985a-a53f3c19c99f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.810190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102eeab-7646-4fcd-b979-4c88127827af admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.814983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.815754 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.816053 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.816372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.817060 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.819048 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.908991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1365/2728] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:59.910033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.912453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.926271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5f8c8-fb3b-42c9-8780-1dd5b3e9650f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.931217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.932515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.932888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.933173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] PATH_INFO: `/v3/projects/a1b1e6abd55a4b17b4a452a80a6bbdbc` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.934496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b1e6abd55a4b17b4a452a80a6bbdbc)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.936441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.956223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:00.011091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e396c6de22e485bb62fcecfd7f49243', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.012023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.012392 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.012713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.014681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.024524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:00.027412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.029268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.040338 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1364/2729] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:00.041734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.052293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cca2e-6287-400d-b102-34acc45ac448 service nova] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.059633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.060365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.060753 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.061129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.061948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.063741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.070550 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b1e6abd55a4b17b4a452a80a6bbdbc', 'request_id': 'req-8bb4a7a4-992b-4f98-add1-221192a54979'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.074652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1365/2730] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:00.077854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b1e6abd55a4b17b4a452a80a6bbdbc', 'request_id': 'req-8bb4a7a4-992b-4f98-add1-221192a54979'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.078837 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.088944 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc48305-619e-43e9-8949-57e1bef1c1ce admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.148010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.148682 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.148967 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.149243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] PATH_INFO: `/v3/projects/adefba353d4841fd914f1aefd5c276e6/users/8415e464c2c4458bba3ccfa4cc9861ac/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.165032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=adefba353d4841fd914f1aefd5c276e6, user_id=8415e464c2c4458bba3ccfa4cc9861ac, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.165254 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b1e6abd55a4b17b4a452a80a6bbdbc', 'request_id': 'req-8bb4a7a4-992b-4f98-add1-221192a54979'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.168547 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.179498 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] The token cache is being invalidate because project a1b1e6abd55a4b17b4a452a80a6bbdbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:00.179890 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.180827 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.182636 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1366/2731] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:59 2022] DELETE /identity/v3/projects/a1b1e6abd55a4b17b4a452a80a6bbdbc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:00.186735 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.188102 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.201911 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a7a4-992b-4f98-add1-221192a54979 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.206797 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.207433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.207719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.208000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] PATH_INFO: `/v3/users/ffaec5dad26d4d0b9625eaf238243cd4` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.209262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffaec5dad26d4d0b9625eaf238243cd4)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.211453 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.291117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1366/2732] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:00 2022] PUT /identity/v3/projects/adefba353d4841fd914f1aefd5c276e6/users/8415e464c2c4458bba3ccfa4cc9861ac/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:00.295808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.297522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.368653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3c081-504f-4b1d-a597-aeb018ae0fdb admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.374342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.374976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.375390 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.375689 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.376374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.377969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.390403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1367/2733] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:00.394408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.405060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487564-7647-4bce-b72d-d657c7932a79 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.411660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.412372 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.412661 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.412942 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] PATH_INFO: `/v3/projects/adefba353d4841fd914f1aefd5c276e6/users/8415e464c2c4458bba3ccfa4cc9861ac/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.415082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adefba353d4841fd914f1aefd5c276e6, user_id=8415e464c2c4458bba3ccfa4cc9861ac, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.418814 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c5b2d-23e5-4ca3-b9ac-9c712104b405 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.426535 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffaec5dad26d4d0b9625eaf238243cd4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.427272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffaec5dad26d4d0b9625eaf238243cd4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.427895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffaec5dad26d4d0b9625eaf238243cd4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.430772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1368/2734] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:00 2022] PUT /identity/v3/projects/adefba353d4841fd914f1aefd5c276e6/users/8415e464c2c4458bba3ccfa4cc9861ac/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:00.442732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.442923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffaec5dad26d4d0b9625eaf238243cd4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.443021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.443106 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.473127 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] MFA Rules not processed for user `8415e464c2c4458bba3ccfa4cc9861ac`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:00.474421 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.545833 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1369/2735] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:00.557521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.559073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.564992 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffaec5dad26d4d0b9625eaf238243cd4'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:00.579360 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1367/2736] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:00 2022] DELETE /identity/v3/users/ffaec5dad26d4d0b9625eaf238243cd4 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:00.623740 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe139-be01-481f-b6a2-c465ebdab78c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.678297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.679092 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.679393 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.679670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.745591 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.747253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.757413 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1370/2737] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:00.855434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.857083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.924360 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:00.927114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.928534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3b2b0-6dac-4d49-aa0f-7bdb2ccf00d0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.928938 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.941172 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.941847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.942132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.942451 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] PATH_INFO: `/v3/users/0b6e1e61fc6845d0afb742152e55774d` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.943737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6e1e61fc6845d0afb742152e55774d)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.945264 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1265fe-68e1-4342-a76f-012a0af2d863 service cinder] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.945728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.951233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.952622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.952622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.952622 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] PATH_INFO: `/v3/projects/9f223cbeefe149efb4ecbb07888c927e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.953572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f223cbeefe149efb4ecbb07888c927e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.956194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.965142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:01.316157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f223cbeefe149efb4ecbb07888c927e', 'request_id': 'req-7a5c9804-5a34-4512-95d2-9b89c505f65e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.316695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6e1e61fc6845d0afb742152e55774d'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.324783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6e1e61fc6845d0afb742152e55774d'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.324783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6e1e61fc6845d0afb742152e55774d'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.325007 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f223cbeefe149efb4ecbb07888c927e', 'request_id': 'req-7a5c9804-5a34-4512-95d2-9b89c505f65e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.328846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f223cbeefe149efb4ecbb07888c927e', 'request_id': 'req-7a5c9804-5a34-4512-95d2-9b89c505f65e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.332652 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6e1e61fc6845d0afb742152e55774d'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.435149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] The token cache is being invalidate because project 9f223cbeefe149efb4ecbb07888c927e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:01.435453 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.436130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-7a5c9804-5a34-4512-95d2-9b89c505f65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.438082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1371/2738] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:00 2022] DELETE /identity/v3/projects/9f223cbeefe149efb4ecbb07888c927e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:01.496759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6e1e61fc6845d0afb742152e55774d'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:01.514036 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1368/2739] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:00 2022] DELETE /identity/v3/users/0b6e1e61fc6845d0afb742152e55774d => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:01.518839 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:01.519185 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:01.519473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:01.546564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:01.547742 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:01.571436 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:01.625224 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1372/2740] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:01.635939 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:01.637720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:01.704310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffd9a-0521-4464-8904-9626287ff4bf admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:01.709051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:01.709692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:01.709977 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:01.710250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:01.721033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:01.722580 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:01.733244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1369/2741] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:02.172871 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.174623 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:02.188488 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb6a02-5755-4ac0-8f32-70030cdfebb5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.193329 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.193991 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.194317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.194599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] PATH_INFO: `/v3/projects/986f7e3ba4ee4cb0904579f1d8342f36` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.195880 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986f7e3ba4ee4cb0904579f1d8342f36)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.198900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.331824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986f7e3ba4ee4cb0904579f1d8342f36', 'request_id': 'req-8bd94c91-0e6c-4814-9ee3-8f12672a8552'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.338474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986f7e3ba4ee4cb0904579f1d8342f36', 'request_id': 'req-8bd94c91-0e6c-4814-9ee3-8f12672a8552'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.344840 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986f7e3ba4ee4cb0904579f1d8342f36', 'request_id': 'req-8bd94c91-0e6c-4814-9ee3-8f12672a8552'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.473649 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] The token cache is being invalidate because project 986f7e3ba4ee4cb0904579f1d8342f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:02.473919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.474572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.477416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1373/2742] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:02 2022] DELETE /identity/v3/projects/986f7e3ba4ee4cb0904579f1d8342f36 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:02.480433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.490416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06f3a8-e576-45dd-8b19-b021e86225a8 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.550594 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.551296 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.551592 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.551906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] PATH_INFO: `/v3/users/8415e464c2c4458bba3ccfa4cc9861ac` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.553099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8415e464c2c4458bba3ccfa4cc9861ac)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.555079 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.802796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:02.810376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8415e464c2c4458bba3ccfa4cc9861ac'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.811083 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8415e464c2c4458bba3ccfa4cc9861ac'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.811756 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8415e464c2c4458bba3ccfa4cc9861ac'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.827284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8415e464c2c4458bba3ccfa4cc9861ac'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.902725 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-63f85bed-dcad-4603-b3a8-1c055f2240f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8415e464c2c4458bba3ccfa4cc9861ac'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.916785 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1370/2743] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:02 2022] DELETE /identity/v3/users/8415e464c2c4458bba3ccfa4cc9861ac => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:03.315824 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.317430 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.343732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94c91-0e6c-4814-9ee3-8f12672a8552 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:03.390507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:03.402298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:03.403013 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.403306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.403587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] PATH_INFO: `/v3/projects/adefba353d4841fd914f1aefd5c276e6` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.404781 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adefba353d4841fd914f1aefd5c276e6)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:03.406763 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:03.537532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adefba353d4841fd914f1aefd5c276e6', 'request_id': 'req-088a2d05-8051-4f25-a782-447e55ef46d6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.543888 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adefba353d4841fd914f1aefd5c276e6', 'request_id': 'req-088a2d05-8051-4f25-a782-447e55ef46d6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.549374 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adefba353d4841fd914f1aefd5c276e6', 'request_id': 'req-088a2d05-8051-4f25-a782-447e55ef46d6'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.621540 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] The token cache is being invalidate because project adefba353d4841fd914f1aefd5c276e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:03.621849 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.622641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-088a2d05-8051-4f25-a782-447e55ef46d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.624575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1374/2744] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:03 2022] DELETE /identity/v3/projects/adefba353d4841fd914f1aefd5c276e6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:03.630328 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.630876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.630994 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.651444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:03.652730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.720750 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1371/2745] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:03.730632 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.731031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.731386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.752535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:03.753882 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.762268 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1375/2746] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:03.766720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.777331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9fc21-7e59-4454-9c51-c4d93cb192ed None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:03.782255 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:03.782956 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.783245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.783523 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.784256 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:03.784634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:03.786275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:03.802860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1372/2747] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:03.803864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.814032 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357defd0-bed6-4702-af7c-1bac261ec347 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:03.819306 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:03.820105 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.820409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.820667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.821353 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:03.823388 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:03.950422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1376/2748] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:03 2022] POST /identity/v3/projects => generated 399 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:03.954464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.965734 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a814110-5966-4558-b29a-9fba067a76d6 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.034003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.034777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.035136 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.035493 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.036291 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.038441 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.132840 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1373/2749] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:03 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:04.137181 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.152592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36330084-92b6-4fd9-bdb2-4a279cb7a3e2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.158294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.159129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.159482 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.159841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.160659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.162621 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.175575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1377/2750] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:04.179787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.193900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fa84b-0836-4f10-9759-7a47daeba92d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.199322 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.200055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.200342 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.200792 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.201669 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.203465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76050e-da85-45ce-9c80-cf4a92f49abe admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.216352 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1374/2751] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:04.220845 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.231843 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb028f59-f86f-490d-ad3d-1109bd783e51 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.237478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.238132 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.238449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.238731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] PATH_INFO: `/v3/projects/a5f824c4c96d43be8e79b66f204a1047/users/ffa84026924042a39f8168c5d5b49b04/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.255069 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f824c4c96d43be8e79b66f204a1047, user_id=ffa84026924042a39f8168c5d5b49b04, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.257724 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.347359 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1378/2752] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:04 2022] PUT /identity/v3/projects/a5f824c4c96d43be8e79b66f204a1047/users/ffa84026924042a39f8168c5d5b49b04/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:04.348068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.348369 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.348650 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.379492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] MFA Rules not processed for user `ffa84026924042a39f8168c5d5b49b04`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:04.382331 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:04.392074 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:04.455358 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1375/2753] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:04.460023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.545701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86f1de-02b6-4196-a61a-e022a28c5153 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.551010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.551746 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.552134 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.552516 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.553602 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.555750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.658186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:04.667870 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1379/2754] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:04 2022] POST /identity/v3/projects => generated 399 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:04.669190 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.680842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1e6c5-d452-4cbc-aac9-b0a82c2b217f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.746578 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.747196 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.747509 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.747822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.748503 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.750634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.848106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1376/2755] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:04 2022] POST /identity/v3/users => generated 306 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:04.852326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.866290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3e8a9-a69b-4307-87e0-09b7029de1e7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.872297 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.873076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.873367 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.873635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.874377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.876016 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.888630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1380/2756] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:04.892700 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.904449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1161e-9ba1-454f-a857-89cb3e72d20f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.912337 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.913231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.913689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.914272 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] PATH_INFO: `/v3/projects/6c1612d3f18144c08cb487d8db4de090/users/0094b4725c024c58a98aef5ec0ef1e42/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.932527 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1612d3f18144c08cb487d8db4de090, user_id=0094b4725c024c58a98aef5ec0ef1e42, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.935297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.028379 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1377/2757] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:04 2022] PUT /identity/v3/projects/6c1612d3f18144c08cb487d8db4de090/users/0094b4725c024c58a98aef5ec0ef1e42/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:05.032432 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.046294 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28f3bb-cafa-4fb3-b9d5-43a71872e65d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.112331 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.113362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.113704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.114090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.115142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.117538 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d114a0f-2cee-41ce-bda6-678212eb0de5 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.129615 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1381/2758] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:05.134089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.145802 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c706b-1020-4cbd-af1b-29608da4142a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.151300 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.158035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.158035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.158035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] PATH_INFO: `/v3/projects/6c1612d3f18144c08cb487d8db4de090/users/0094b4725c024c58a98aef5ec0ef1e42/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.158035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1612d3f18144c08cb487d8db4de090, user_id=0094b4725c024c58a98aef5ec0ef1e42, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.158035 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.170217 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1378/2759] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:05 2022] PUT /identity/v3/projects/6c1612d3f18144c08cb487d8db4de090/users/0094b4725c024c58a98aef5ec0ef1e42/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:05.178737 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.179096 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.179386 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.209109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] MFA Rules not processed for user `0094b4725c024c58a98aef5ec0ef1e42`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:05.210382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:05.289475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1382/2760] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:05.304064 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.305810 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:05.371108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb95076-8ab1-454e-bc98-0a1c195261c3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.374928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.378416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.378416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.378416 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.386629 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.388084 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.407269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1379/2761] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:05.631341 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.641935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3e0a-1769-4c96-aae7-b40b428344cc None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.645865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.646512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.646804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.647078 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.709643 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.711140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b491773-0a46-489a-8ae1-ac1b2797d3c2 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.721895 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1383/2762] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:10.992141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:11.001173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:11.055784 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474fba-f64c-43a1-9f7f-39c0395ac7a1 service cinder] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:11.060593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:11.061228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:11.061511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:11.061787 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] PATH_INFO: `/v3/users/ffa84026924042a39f8168c5d5b49b04` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:11.062980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa84026924042a39f8168c5d5b49b04)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:11.065010 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:11.327715 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa84026924042a39f8168c5d5b49b04'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:11.328389 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa84026924042a39f8168c5d5b49b04'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:11.334702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa84026924042a39f8168c5d5b49b04'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:11.341975 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa84026924042a39f8168c5d5b49b04'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:11.444758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa84026924042a39f8168c5d5b49b04'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:11.458662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1380/2763] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:10 2022] DELETE /identity/v3/users/ffa84026924042a39f8168c5d5b49b04 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:11.463333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:11.463816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:11.463941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:11.483448 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:11.484669 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:11.488976 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=100 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:11.545090 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1384/2764] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:11.554814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:11.556259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:11.616376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e17af-dc1b-478c-9633-b42cbc1040b0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:11.620654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:11.621271 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:11.621557 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:11.621842 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:11.630991 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:11.632439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:11.642914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1381/2765] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:12.091831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.093566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.106546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89be2c-829d-4299-a116-530bbf67ddf5 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.111241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.111968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.112254 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.112653 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] PATH_INFO: `/v3/projects/a5f824c4c96d43be8e79b66f204a1047` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.113709 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f824c4c96d43be8e79b66f204a1047)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.115817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.240293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f824c4c96d43be8e79b66f204a1047', 'request_id': 'req-c3b5a199-04e1-44a7-bc44-ec311956a1a8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.246867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f824c4c96d43be8e79b66f204a1047', 'request_id': 'req-c3b5a199-04e1-44a7-bc44-ec311956a1a8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.252463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f824c4c96d43be8e79b66f204a1047', 'request_id': 'req-c3b5a199-04e1-44a7-bc44-ec311956a1a8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.358245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] The token cache is being invalidate because project a5f824c4c96d43be8e79b66f204a1047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:12.358474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.359114 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.360631 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1385/2766] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:12 2022] DELETE /identity/v3/projects/a5f824c4c96d43be8e79b66f204a1047 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:12.364822 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.375096 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea875-a127-4c6c-8854-63838fede675 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.408471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:12.439599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.440283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.440657 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.440934 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] PATH_INFO: `/v3/users/0094b4725c024c58a98aef5ec0ef1e42` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.442108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0094b4725c024c58a98aef5ec0ef1e42)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.444187 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.664835 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0094b4725c024c58a98aef5ec0ef1e42'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.665410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0094b4725c024c58a98aef5ec0ef1e42'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.666068 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0094b4725c024c58a98aef5ec0ef1e42'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.679357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0094b4725c024c58a98aef5ec0ef1e42'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.758634 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-e6cba1a7-1ac1-4836-968d-99bbb7fcde2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0094b4725c024c58a98aef5ec0ef1e42'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.772430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1382/2767] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:12 2022] DELETE /identity/v3/users/0094b4725c024c58a98aef5ec0ef1e42 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:13.228846 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.231191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.244258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5a199-04e1-44a7-bc44-ec311956a1a8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.301710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.302507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.302915 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.303197 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] PATH_INFO: `/v3/projects/6c1612d3f18144c08cb487d8db4de090` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.304344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1612d3f18144c08cb487d8db4de090)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.306307 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.419062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:13.428290 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1612d3f18144c08cb487d8db4de090', 'request_id': 'req-ea6942be-6c80-4575-af8e-5a8ce464e00d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.434668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1612d3f18144c08cb487d8db4de090', 'request_id': 'req-ea6942be-6c80-4575-af8e-5a8ce464e00d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.440144 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1612d3f18144c08cb487d8db4de090', 'request_id': 'req-ea6942be-6c80-4575-af8e-5a8ce464e00d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.525530 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] The token cache is being invalidate because project 6c1612d3f18144c08cb487d8db4de090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:13.526474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.527131 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ea6942be-6c80-4575-af8e-5a8ce464e00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.529484 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1386/2768] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:13 2022] DELETE /identity/v3/projects/6c1612d3f18144c08cb487d8db4de090 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:13.535343 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.535877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.535877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.555704 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:13.556995 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.617430 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1383/2769] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:13.626326 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.626644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.626923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.646975 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:13.648250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.657146 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1387/2770] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:13.661480 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.672607 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77838ef-1aa6-4103-8cfe-8e42354832ac None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.677236 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.677908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.678192 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.678513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.679208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:13.679515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.681138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.693703 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1384/2771] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:13.701741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.711293 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d74f-de54-44c4-be4c-b88ef3f3b00a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.716251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.716908 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.717189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.717463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.718411 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.720346 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.805142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1388/2772] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:13.809815 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.820078 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e4dc6-9aed-4ee2-ba74-6bafb0bf5f9d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.854877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:13.879847 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.880471 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.880758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.881037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.881702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.883684 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.999492 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1385/2773] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:14.003836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.015537 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ee89c-9fd9-4ca6-a24d-a93eca3cc17f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.020574 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.021194 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.021478 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.021815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.022585 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.024186 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.035038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1389/2774] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.039231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.050753 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79516e34-5857-4cad-b7c4-3081138515e2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.055588 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.056131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.056418 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.056699 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.057446 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.059119 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492968-e7b9-44e8-a2e1-a1671fb272f0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.069351 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1386/2775] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.074116 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.084154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830e728-8a40-46e8-ad0e-2ab56557a4ea admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.089443 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.090093 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.090401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.090684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] PATH_INFO: `/v3/projects/daf18ccf81a74c4385718677b6abf673/users/80da0309d45c4796b102da820a53f27d/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.105553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf18ccf81a74c4385718677b6abf673, user_id=80da0309d45c4796b102da820a53f27d, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.108252 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.193206 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1390/2776] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:14 2022] PUT /identity/v3/projects/daf18ccf81a74c4385718677b6abf673/users/80da0309d45c4796b102da820a53f27d/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:14.204413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.204766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.205051 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.236542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] MFA Rules not processed for user `80da0309d45c4796b102da820a53f27d`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:14.237906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.304175 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1387/2777] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:14.308948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.371107 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e83afb-2b1d-432f-b7b4-6e56f99fca75 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.376251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.376841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.377122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.377399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.378067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.380046 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.486732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1391/2778] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:14.491301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.503214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464923a7-0d22-4af8-aa4a-7b5d209a0062 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.562763 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.563444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.563563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.563832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.564505 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.566434 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.668481 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1388/2779] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:14.672691 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.683415 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8120b-9bab-4fc8-9721-56150c8bbf8f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.688245 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.689047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.689330 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.694446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.694446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.694446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.704028 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1392/2780] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.708310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.719382 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1438d-eefb-4c26-ba01-3271fa476324 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.724223 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.724908 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.725195 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.725476 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] PATH_INFO: `/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5/users/9f255ff29d224832abba4dfb1f697a7f/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.740933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dd52db39a34c05b38ba22a7b4b71d5, user_id=9f255ff29d224832abba4dfb1f697a7f, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.743633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.856228 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1389/2781] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:14 2022] PUT /identity/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5/users/9f255ff29d224832abba4dfb1f697a7f/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:14.860256 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.870923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e0297-9789-4554-b13d-a23d0e6857d8 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.902068 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.928284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.928974 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.929253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.929553 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.930242 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.931858 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84f731-e22d-42a5-a7cb-3ba16d6e4197 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.942246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1393/2782] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.946205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.958213 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42940fe8-8732-4ce5-993e-2e95bcaf2c72 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.963826 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.964495 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.964779 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.965059 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] PATH_INFO: `/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5/users/9f255ff29d224832abba4dfb1f697a7f/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.967105 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dd52db39a34c05b38ba22a7b4b71d5, user_id=9f255ff29d224832abba4dfb1f697a7f, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.970056 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.973435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.981077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1390/2783] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:14 2022] PUT /identity/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5/users/9f255ff29d224832abba4dfb1f697a7f/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:14.992250 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.992543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.992825 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.020680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] MFA Rules not processed for user `9f255ff29d224832abba4dfb1f697a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:15.021889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.091396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1394/2784] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:15.102543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.104502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.171207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48758bb-3485-4a8e-828c-8a9c5b1eb2da admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.174998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.175630 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.175944 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.176224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.237207 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.238702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87099d-3aeb-47e4-93d4-5fe3d558ec4d service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.248354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1391/2785] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:18.387572 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.449552 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350c76c-67f7-4ac1-b7dc-ee9b5321504e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.454123 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.454780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.455067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.455347 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] PATH_INFO: `/v3/users/80da0309d45c4796b102da820a53f27d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.456925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80da0309d45c4796b102da820a53f27d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.458864 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.692491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80da0309d45c4796b102da820a53f27d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:18.693129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80da0309d45c4796b102da820a53f27d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:18.799527 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80da0309d45c4796b102da820a53f27d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:18.801126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:18.812177 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80da0309d45c4796b102da820a53f27d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:18.812900 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80da0309d45c4796b102da820a53f27d'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:18.825500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1395/2786] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:18 2022] DELETE /identity/v3/users/80da0309d45c4796b102da820a53f27d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:18.830020 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.830536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.830644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.851041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:18.852587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:18.913406 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1392/2787] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:18.923984 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.925396 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:18.987502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da95e6d-0702-411b-9bf8-138b900f0109 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.991060 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.991671 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.991949 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.992226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.001968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.003441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.013759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1396/2788] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:19.596887 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.597087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:19.609245 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16463ec3-44fa-4518-8175-f64822757d1e None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.613878 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.614555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.614834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.615106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] PATH_INFO: `/v3/projects/daf18ccf81a74c4385718677b6abf673` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.616292 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf18ccf81a74c4385718677b6abf673)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.618166 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.722243 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:19.725137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf18ccf81a74c4385718677b6abf673', 'request_id': 'req-0945e08c-1473-44cf-aa8c-746d0e52c345'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:19.730864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf18ccf81a74c4385718677b6abf673', 'request_id': 'req-0945e08c-1473-44cf-aa8c-746d0e52c345'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:19.844546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf18ccf81a74c4385718677b6abf673', 'request_id': 'req-0945e08c-1473-44cf-aa8c-746d0e52c345'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:19.856491 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] The token cache is being invalidate because project daf18ccf81a74c4385718677b6abf673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:19.856783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:19.857443 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:19.859006 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1393/2789] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:19 2022] DELETE /identity/v3/projects/daf18ccf81a74c4385718677b6abf673 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:19.863149 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.873403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16aec95-b241-44f8-a92a-cc86c58306f9 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.930447 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.931091 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.931378 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.932190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] PATH_INFO: `/v3/users/9f255ff29d224832abba4dfb1f697a7f` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.933461 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f255ff29d224832abba4dfb1f697a7f)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.935491 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:20.115486 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f255ff29d224832abba4dfb1f697a7f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.116067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f255ff29d224832abba4dfb1f697a7f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.198797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f255ff29d224832abba4dfb1f697a7f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.211463 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f255ff29d224832abba4dfb1f697a7f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.212130 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-ab581867-8f60-485c-92db-3cb140d547ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f255ff29d224832abba4dfb1f697a7f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.218401 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:20.225713 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1397/2790] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:19 2022] DELETE /identity/v3/users/9f255ff29d224832abba4dfb1f697a7f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:20.731737 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:20.736269 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:20.750405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e08c-1473-44cf-aa8c-746d0e52c345 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:20.810439 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:20.811088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:20.811376 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:20.811660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] PATH_INFO: `/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:20.812839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36dd52db39a34c05b38ba22a7b4b71d5)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:20.814791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:20.941174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36dd52db39a34c05b38ba22a7b4b71d5', 'request_id': 'req-1d897966-5529-40e7-9aed-72953d95e56b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:20.946706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36dd52db39a34c05b38ba22a7b4b71d5', 'request_id': 'req-1d897966-5529-40e7-9aed-72953d95e56b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:21.049262 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36dd52db39a34c05b38ba22a7b4b71d5', 'request_id': 'req-1d897966-5529-40e7-9aed-72953d95e56b'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:21.061989 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] The token cache is being invalidate because project 36dd52db39a34c05b38ba22a7b4b71d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:21.062281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:21.062950 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-1d897966-5529-40e7-9aed-72953d95e56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:21.064478 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1394/2791] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:20 2022] DELETE /identity/v3/projects/36dd52db39a34c05b38ba22a7b4b71d5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:21.069400 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.069893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.070005 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.090750 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:21.091968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:21.151577 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1398/2792] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:21.159554 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.159846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.160135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.180604 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:21.182716 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:21.190919 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1395/2793] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:21.195288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.205520 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02848a0f-9ef3-43a4-adba-3fe96beee725 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.210507 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.211128 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.211418 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.211710 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.212428 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:21.213371 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.215018 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.227034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1399/2794] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:21.232085 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.241806 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52267-7f90-43a4-b08d-ae0cb144dc7f None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.246798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.247405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.247691 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.247968 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.248633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.250564 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.377273 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1396/2795] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:21 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:21.381700 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.391953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af252-ecc7-4465-9f1a-798bcdbab434 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.456072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.456707 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.457011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.457283 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.457955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.459980 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.626759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1400/2796] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:21 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:21.630986 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.642174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12de39-1418-4e5f-8b1a-a80f9fcfa692 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.646803 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.647403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.647682 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.647957 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.648639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.650938 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.660971 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1397/2797] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:21.664996 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.675587 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75447369-e8ee-4139-b04a-13c2a7b356a0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.680237 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.680847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.681156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.681476 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.682201 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.683841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907d4c9-788d-42ac-8198-c986d46ef4aa admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.695948 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1401/2798] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:21.700697 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:21.710914 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636344ca-c914-4957-93a3-c2c304e31242 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:21.716611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:21.717456 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.717724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.718305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] PATH_INFO: `/v3/projects/5adac42cbbce41ce8501ff49f5ec407f/users/950e81ac2d1d4a18a38ef8f9ac755880/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.733341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5adac42cbbce41ce8501ff49f5ec407f, user_id=950e81ac2d1d4a18a38ef8f9ac755880, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:21.736150 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:21.822963 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:21.828666 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1398/2799] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:21 2022] PUT /identity/v3/projects/5adac42cbbce41ce8501ff49f5ec407f/users/950e81ac2d1d4a18a38ef8f9ac755880/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:21.840129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:21.840355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:21.840636 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:21.869259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] MFA Rules not processed for user `950e81ac2d1d4a18a38ef8f9ac755880`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:21.871204 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:21.920500 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:21.941475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1402/2800] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:22.047772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:22.047772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:22.047772 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501aaa9-b3f9-4c2b-941e-30ed476e87b0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:22.049125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:22.049125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:22.049125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:22.049125 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:22.049631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:22.049631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f73770-33c8-4363-a508-19b0767ff6b4 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:22.049631 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1399/2801] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:32.106159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=26 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:32.136165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=4 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:33.795576 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:33.863047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ef396-e19a-4df3-8ebb-fc6c4e6de552 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:33.867773 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:33.868427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:33.868741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:33.869017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] PATH_INFO: `/v3/users/950e81ac2d1d4a18a38ef8f9ac755880` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:33.870189 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=950e81ac2d1d4a18a38ef8f9ac755880)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:33.872159 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.115071 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950e81ac2d1d4a18a38ef8f9ac755880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.115760 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950e81ac2d1d4a18a38ef8f9ac755880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.215366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950e81ac2d1d4a18a38ef8f9ac755880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.229209 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950e81ac2d1d4a18a38ef8f9ac755880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.229903 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950e81ac2d1d4a18a38ef8f9ac755880'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.242835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1403/2802] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:33 2022] DELETE /identity/v3/users/950e81ac2d1d4a18a38ef8f9ac755880 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:34.248014 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.248552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.248936 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.268601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:34.269805 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.336131 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1400/2803] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:34.346457 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.347894 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.407579 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849abecf-6dbb-4c67-9563-0d45c872b81d admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.411399 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.412017 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.412328 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.412608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.422524 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.423968 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738ee22-9145-4132-ab3d-8deacb07567c service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.433608 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1404/2804] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:34.930204 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.931807 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.945123 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a8fe1-1f61-4bbc-9f46-af22f11124ab None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.949708 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.950881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.951193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.951694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] PATH_INFO: `/v3/projects/5adac42cbbce41ce8501ff49f5ec407f` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.953232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5adac42cbbce41ce8501ff49f5ec407f)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.959758 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.057796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5adac42cbbce41ce8501ff49f5ec407f', 'request_id': 'req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.063846 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5adac42cbbce41ce8501ff49f5ec407f', 'request_id': 'req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.151529 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5adac42cbbce41ce8501ff49f5ec407f', 'request_id': 'req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.165413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] The token cache is being invalidate because project 5adac42cbbce41ce8501ff49f5ec407f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:35.165709 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.166405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-2ea21b5f-bb2e-49b7-a214-016093bcdfe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.168021 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1401/2805] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:34 2022] DELETE /identity/v3/projects/5adac42cbbce41ce8501ff49f5ec407f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:35.179142 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.179667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.179804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.200084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:35.201866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.220342 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:35.263262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1405/2806] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:35.271694 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.272212 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.272323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.296174 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:35.297760 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.306440 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1402/2807] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:35.311619 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.321899 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b96ea-05a4-4af6-aa7a-61549ab4431d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.327652 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.328490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.328812 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.329190 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.329909 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:35.330217 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.331847 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.344754 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1406/2808] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:35.349225 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.363222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d9ae8-1f31-4747-9b62-ce0cea52e4d3 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.367856 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.368499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.368740 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.368998 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.369671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.371599 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.486238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1403/2809] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:35.490522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.501526 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a413e-4fba-44b5-abc4-80470968f8f0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.562156 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.562800 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.563084 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.563370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.564023 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.565943 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.683317 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1407/2810] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:35.688340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.698673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d29e05-1427-4b7f-ba13-2198ca4c9aef admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.703658 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c623-f852-4423-9005-82d76544511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.704265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c623-f852-4423-9005-82d76544511a admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.704542 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c623-f852-4423-9005-82d76544511a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.704818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c623-f852-4423-9005-82d76544511a admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.705502 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c623-f852-4423-9005-82d76544511a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.707135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c623-f852-4423-9005-82d76544511a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.717205 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1404/2811] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:35.726169 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.737772 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dae6-34c2-4250-ac2f-2b4066c4f3e7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.742433 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.743040 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.743323 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.743605 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.744295 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.745884 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983da84-2951-4a5f-9460-9c67cff9004e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.757776 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1408/2812] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:35.762601 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c623-f852-4423-9005-82d76544511a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.777373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c623-f852-4423-9005-82d76544511a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.782882 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.783552 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.783834 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.784109 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] PATH_INFO: `/v3/projects/6d8a4598dd0741fabc1b38917626e133/users/965acd6b6b44472ebf6511f4cbb54b45/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.800075 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8a4598dd0741fabc1b38917626e133, user_id=965acd6b6b44472ebf6511f4cbb54b45, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.802917 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.890244 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1405/2813] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:35 2022] PUT /identity/v3/projects/6d8a4598dd0741fabc1b38917626e133/users/965acd6b6b44472ebf6511f4cbb54b45/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:35.900741 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4810-6fa4-4377-b241-39d2b312f063 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.901025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4810-6fa4-4377-b241-39d2b312f063 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.901313 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a4810-6fa4-4377-b241-39d2b312f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.929926 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9d9a4810-6fa4-4377-b241-39d2b312f063 None None] MFA Rules not processed for user `965acd6b6b44472ebf6511f4cbb54b45`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:35.931221 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9d9a4810-6fa4-4377-b241-39d2b312f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.999379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1409/2814] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:36.010662 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:36.012545 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:36.035139 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:36.072619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00973ffe-e663-4337-a0ea-674d2237210d admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:36.076477 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:36.077091 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:36.077373 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:36.077648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:36.086159 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:36.087674 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4694d04-bae8-4c97-998f-8346b29edc33 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:36.097900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1406/2815] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:37.816255 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.816841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.816841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.838089 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] MFA Rules not processed for user `965acd6b6b44472ebf6511f4cbb54b45`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:37.839355 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:37.847000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1410/2816] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:46.409431 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=17 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:47.831320 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:59.248392 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7923c6-06c1-47ff-8318-a1dd0e73bd5d None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.248949 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7923c6-06c1-47ff-8318-a1dd0e73bd5d None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.249057 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7923c6-06c1-47ff-8318-a1dd0e73bd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.301402 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7d7923c6-06c1-47ff-8318-a1dd0e73bd5d None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:59.302695 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7d7923c6-06c1-47ff-8318-a1dd0e73bd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:59.371866 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1407/2817] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:14:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:59.523366 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:59.524856 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:59.645912 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3c0d0-11e4-4911-add9-31959542c03a None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:59.653797 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:59.654474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.654755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.655036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.665195 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:59.667039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:59.676171 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1411/2818] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:00.613948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9bcdd-18c8-43e6-b2cc-b9c15197bf42 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:00.614513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9bcdd-18c8-43e6-b2cc-b9c15197bf42 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:00.614647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9bcdd-18c8-43e6-b2cc-b9c15197bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:00.634293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-43a9bcdd-18c8-43e6-b2cc-b9c15197bf42 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:00.635549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-43a9bcdd-18c8-43e6-b2cc-b9c15197bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:00.643900 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1408/2819] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:15:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:00.656036 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:00.667522 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0650fea-980f-4064-a103-2f703eeec7bc service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:00.673157 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:00.673771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:00.674057 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:00.674335 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:00.683045 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:00.684670 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:00.693595 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1412/2820] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:15:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:01.438512 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa3621-11e5-455f-b5ee-39eff788f53e None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:01.438796 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa3621-11e5-455f-b5ee-39eff788f53e None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:01.439129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa3621-11e5-455f-b5ee-39eff788f53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:01.463121 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-a3aa3621-11e5-455f-b5ee-39eff788f53e None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:01.464332 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a3aa3621-11e5-455f-b5ee-39eff788f53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:01.472980 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1409/2821] 10.222.0.85 () {58 vars in 1009 bytes} [Thu Jul 28 18:15:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:01.487897 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:01.497950 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78f63-7b8c-41b1-904b-f7bbc3818d2c service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:01.503969 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:01.506191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:01.506191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:01.506191 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:01.514225 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:01.516548 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:01.525594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1413/2822] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:15:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:02.207928 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:02.208403 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:02.208549 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:02.230170 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] MFA Rules not processed for user `81393bdb74d84cfc886bdf47c2512d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:02.231415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:02.239789 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1410/2823] 10.222.0.85 () {58 vars in 967 bytes} [Thu Jul 28 18:15:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:02.252964 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:02.263925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88041600-fa9c-42ba-ba18-e6d7545a39c2 service swift] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:02.269835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5ce51b2ba94ce3ad65f1d61c3efe60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:02.270544 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:02.270835 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:02.272286 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:02.279953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:02.281799 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec63c-cb46-4148-8684-4bbf863d0c99 service swift] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:02.290138 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1414/2824] 10.222.0.85 () {60 vars in 1384 bytes} [Thu Jul 28 18:15:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:05.200283 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:05.270115 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fab3ab-bada-46e3-8e46-d71457a2b244 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:05.274687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:05.275348 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:05.275624 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:05.275909 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] PATH_INFO: `/v3/users/965acd6b6b44472ebf6511f4cbb54b45` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:05.277488 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=965acd6b6b44472ebf6511f4cbb54b45)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:05.280336 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:05.500724 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965acd6b6b44472ebf6511f4cbb54b45'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:05.501499 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965acd6b6b44472ebf6511f4cbb54b45'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:05.502138 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965acd6b6b44472ebf6511f4cbb54b45'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:05.508818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:05.517730 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965acd6b6b44472ebf6511f4cbb54b45'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:05.608804 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965acd6b6b44472ebf6511f4cbb54b45'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:05.622232 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1411/2825] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:05 2022] DELETE /identity/v3/users/965acd6b6b44472ebf6511f4cbb54b45 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:05.627427 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:05.628021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:05.628021 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:05.647759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:05.649226 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:05.712278 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1415/2826] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:05.723250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:05.724726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:05.784264 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d37cd9-dbb4-4f4d-97bc-8467d42bd3f3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:05.788140 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:05.788798 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:05.789104 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:05.789409 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:05.799597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:05.801041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:05.811088 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1412/2827] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:06.080241 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:06.082743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:06.111745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cc30-572d-49ec-a939-94458c141330 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:06.121717 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:06.122449 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:06.122738 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:06.123202 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] PATH_INFO: `/v3/projects/6d8a4598dd0741fabc1b38917626e133` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:06.124403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8a4598dd0741fabc1b38917626e133)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:06.126945 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:06.234133 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:06.254426 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8a4598dd0741fabc1b38917626e133', 'request_id': 'req-45993209-630c-4df8-99f1-c99fbfc1297f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:06.261049 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8a4598dd0741fabc1b38917626e133', 'request_id': 'req-45993209-630c-4df8-99f1-c99fbfc1297f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:06.267097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8a4598dd0741fabc1b38917626e133', 'request_id': 'req-45993209-630c-4df8-99f1-c99fbfc1297f'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:06.383103 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] The token cache is being invalidate because project 6d8a4598dd0741fabc1b38917626e133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:06.383521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:06.384456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:06.386625 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1416/2828] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:06 2022] DELETE /identity/v3/projects/6d8a4598dd0741fabc1b38917626e133 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:08.444231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:08.445687 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:08.461106 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f19f7-c6b2-443e-99e4-95b3715687ed service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:08.519660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:08.520354 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:08.520659 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:08.521323 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] PATH_INFO: `/v3/users/de20d8b31db44abf91a482d81b898755` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:08.522179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de20d8b31db44abf91a482d81b898755)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:08.524278 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:08.789664 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de20d8b31db44abf91a482d81b898755'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:08.790318 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de20d8b31db44abf91a482d81b898755'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:08.790999 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de20d8b31db44abf91a482d81b898755'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:08.804117 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de20d8b31db44abf91a482d81b898755'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:08.871015 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-edbc5905-48b7-4672-bfb9-456e1868d10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de20d8b31db44abf91a482d81b898755'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:08.877639 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:08.890263 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1413/2829] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:08 2022] DELETE /identity/v3/users/de20d8b31db44abf91a482d81b898755 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:09.227382 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:09.228854 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:09.243281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45993209-630c-4df8-99f1-c99fbfc1297f admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:09.301343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:09.302010 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.302298 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.302609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] PATH_INFO: `/v3/projects/b8daf9e3c7be417c82e7cbf8624d1d5d` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.303770 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8daf9e3c7be417c82e7cbf8624d1d5d)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:09.305681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:09.422085 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8daf9e3c7be417c82e7cbf8624d1d5d', 'request_id': 'req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:09.428391 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8daf9e3c7be417c82e7cbf8624d1d5d', 'request_id': 'req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:09.434104 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8daf9e3c7be417c82e7cbf8624d1d5d', 'request_id': 'req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:09.533779 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] The token cache is being invalidate because project b8daf9e3c7be417c82e7cbf8624d1d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:09.534073 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:09.534728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-bf63c9f1-c3ab-4a54-bf54-6ceb6b65e08a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:09.536251 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1417/2830] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:09 2022] DELETE /identity/v3/projects/b8daf9e3c7be417c82e7cbf8624d1d5d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:09.541462 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.542000 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.542108 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.562884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:09.564129 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:09.627371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1414/2831] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:09.635546 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791878-4361-4f38-abb1-80b400170a4c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.635841 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791878-4361-4f38-abb1-80b400170a4c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.636122 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79791878-4361-4f38-abb1-80b400170a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.657043 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-79791878-4361-4f38-abb1-80b400170a4c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:09.658296 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-79791878-4361-4f38-abb1-80b400170a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:09.666667 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1418/2832] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:09.670962 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:09.681611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76b401-0368-4808-8dbe-9ce564b524d6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:09.686302 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:09.686974 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.687885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.688171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.688877 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:09.689185 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:09.690839 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:09.702520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1415/2833] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:09.707006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79791878-4361-4f38-abb1-80b400170a4c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:09.716935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79791878-4361-4f38-abb1-80b400170a4c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:09.722279 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:09.727806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.727806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.727806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.727806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:09.727806 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:09.863041 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1419/2834] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:09 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:09.867341 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:09.877620 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd859a-9715-4100-9fcb-26bc53d93b3f admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:09.940616 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:09.941506 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.941990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.942531 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.943767 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:09.947127 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.037577 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1416/2835] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:09 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:10.041808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.054802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf603-e293-4f70-a97b-8a02d71645cc admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.059560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.060172 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.060456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.060739 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.061441 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.063205 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.073541 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1420/2836] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:15:10.077673 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.088349 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8789d9-edbf-465f-834b-7dc13ec98d23 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.093357 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.094563 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.094850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.095130 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.095817 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.097410 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8f7c8-5068-4ea9-8a28-5f03dda3a876 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.107943 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1417/2837] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:15:10.112459 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.119704 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:10.123768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55aeef-458e-4bc8-8ee8-476b60b0a4de admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.129610 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.130282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.134651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.134651 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] PATH_INFO: `/v3/projects/630dc0c9dc1746e4b46a5f692328b04b/users/0a67f1acf5584b15b0bd9f3ff3cfb52e/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.146474 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630dc0c9dc1746e4b46a5f692328b04b, user_id=0a67f1acf5584b15b0bd9f3ff3cfb52e, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.149160 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.220728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1421/2838] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:10 2022] PUT /identity/v3/projects/630dc0c9dc1746e4b46a5f692328b04b/users/0a67f1acf5584b15b0bd9f3ff3cfb52e/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:10.231120 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.231411 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.231713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.259310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] MFA Rules not processed for user `0a67f1acf5584b15b0bd9f3ff3cfb52e`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:10.260539 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:10.314421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:10.339780 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1418/2839] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:10.346343 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.421634 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964cf47-2364-47db-87ff-ead3193b5084 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.426423 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.434097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.434097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.434097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.434097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.434097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.678802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1422/2840] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:10 2022] POST /identity/v3/projects => generated 373 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:10.682985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.696489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43189a-20c9-4b1d-b425-a3c7787e1bef None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.786660 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.787363 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.787656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.787964 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.788646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.790640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.878239 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1419/2841] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:10 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:10.882273 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.892881 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5805-8c70-417b-b786-3f1617cac10a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.897452 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.898451 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.898755 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.899035 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.899734 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.901333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:10.912594 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1423/2842] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:10.916867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:10.927904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca6df-3f19-4b57-b18f-6362c034f105 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:10.932672 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:10.933266 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:10.933547 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:10.933825 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] PATH_INFO: `/v3/projects/e046e84c087f4988b940f48a96faf53c/users/be94ba1d43774070970a4e4cd6cc3bfc/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:10.951197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e046e84c087f4988b940f48a96faf53c, user_id=be94ba1d43774070970a4e4cd6cc3bfc, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:10.954128 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:11.032141 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1420/2843] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:10 2022] PUT /identity/v3/projects/e046e84c087f4988b940f48a96faf53c/users/be94ba1d43774070970a4e4cd6cc3bfc/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:11.036243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:11.046955 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7526-f7fa-41e0-a97f-ea2ab44205f7 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:11.105165 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:11.105783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:11.106070 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:11.106370 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:11.107079 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:11.108685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be26ac-672e-4bbf-9230-e7904b6e5346 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:11.119140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1424/2844] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:11.123414 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:11.135025 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489424c-83e0-4f25-b4f2-3a019e6dfad0 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:11.139871 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:11.140640 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:11.140948 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:11.141310 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] PATH_INFO: `/v3/projects/e046e84c087f4988b940f48a96faf53c/users/be94ba1d43774070970a4e4cd6cc3bfc/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:11.143419 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e046e84c087f4988b940f48a96faf53c, user_id=be94ba1d43774070970a4e4cd6cc3bfc, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:11.146093 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:11.157137 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1421/2845] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:11 2022] PUT /identity/v3/projects/e046e84c087f4988b940f48a96faf53c/users/be94ba1d43774070970a4e4cd6cc3bfc/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:11.167635 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:11.167923 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:11.168203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:11.197757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] MFA Rules not processed for user `be94ba1d43774070970a4e4cd6cc3bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:11.198920 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:11.239545 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:11.268067 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1425/2846] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:11.279197 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:11.281421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:11.342399 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38461bf9-3f22-4ada-bdc5-a53596c4c0e7 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:11.346736 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:11.347356 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:11.347641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:11.347916 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:11.356923 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:11.358371 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68073d54-cc10-4643-b199-f167445a8b80 service cinder] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:11.368415 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1422/2847] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:12.374377 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:12.436120 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9ed92-f7c6-4bc8-8394-b4962918d47d None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:12.441387 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:12.442025 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:12.442376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:12.442681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] PATH_INFO: `/v3/users/0a67f1acf5584b15b0bd9f3ff3cfb52e` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:12.443919 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a67f1acf5584b15b0bd9f3ff3cfb52e)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:12.445850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:12.581935 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a67f1acf5584b15b0bd9f3ff3cfb52e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:12.586783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a67f1acf5584b15b0bd9f3ff3cfb52e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:12.716831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a67f1acf5584b15b0bd9f3ff3cfb52e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:12.730745 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a67f1acf5584b15b0bd9f3ff3cfb52e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:12.731490 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a67f1acf5584b15b0bd9f3ff3cfb52e'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:12.744638 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1426/2848] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:12 2022] DELETE /identity/v3/users/0a67f1acf5584b15b0bd9f3ff3cfb52e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:12.749176 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:12.749726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:12.749850 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:12.768576 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:12.769906 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:12.810390 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:12.829398 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1423/2849] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:12.838961 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:12.840505 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:12.898262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035d56f-afb9-4d76-9562-6e274c07dd86 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:12.901941 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:12.902665 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:12.902953 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:12.903243 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:12.913244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:12.915082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:12.919842 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:12.925743 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1427/2850] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:13.374089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:13.376340 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:13.388544 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721745d-42f5-4591-bda5-34ee4758eaf2 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:13.393525 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:13.394178 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:13.394574 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:13.394855 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] PATH_INFO: `/v3/projects/630dc0c9dc1746e4b46a5f692328b04b` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:13.396055 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=630dc0c9dc1746e4b46a5f692328b04b)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:13.398033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:13.505250 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630dc0c9dc1746e4b46a5f692328b04b', 'request_id': 'req-38d70157-e5f0-49c7-8ff2-7f18e728c03e'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.510783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630dc0c9dc1746e4b46a5f692328b04b', 'request_id': 'req-38d70157-e5f0-49c7-8ff2-7f18e728c03e'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.583777 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630dc0c9dc1746e4b46a5f692328b04b', 'request_id': 'req-38d70157-e5f0-49c7-8ff2-7f18e728c03e'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.596841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] The token cache is being invalidate because project 630dc0c9dc1746e4b46a5f692328b04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:13.604494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.604494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.604494 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1424/2851] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:13 2022] DELETE /identity/v3/projects/630dc0c9dc1746e4b46a5f692328b04b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:13.604829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:13.614531 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db54a5a-bcef-4b80-ace4-4f0b5085e4d4 service neutron] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:13.676558 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:13.677117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:13.677404 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:13.677681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] PATH_INFO: `/v3/users/be94ba1d43774070970a4e4cd6cc3bfc` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:13.678893 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be94ba1d43774070970a4e4cd6cc3bfc)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:13.680804 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:13.871560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be94ba1d43774070970a4e4cd6cc3bfc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.872219 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be94ba1d43774070970a4e4cd6cc3bfc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.944475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be94ba1d43774070970a4e4cd6cc3bfc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.958129 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be94ba1d43774070970a4e4cd6cc3bfc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.964680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-3a8b0442-7d33-43db-abfa-42fe91164e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be94ba1d43774070970a4e4cd6cc3bfc'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:13.972829 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1428/2852] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:13 2022] DELETE /identity/v3/users/be94ba1d43774070970a4e4cd6cc3bfc => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:14.655334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:14.666484 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:14.670648 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70157-e5f0-49c7-8ff2-7f18e728c03e admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:14.733206 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:14.734433 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:14.734717 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:14.735208 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] PATH_INFO: `/v3/projects/e046e84c087f4988b940f48a96faf53c` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:14.754459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e046e84c087f4988b940f48a96faf53c)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:14.754459 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:14.990641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e046e84c087f4988b940f48a96faf53c', 'request_id': 'req-43f44072-e501-4ce9-8252-be4b618d6e78'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:14.996145 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e046e84c087f4988b940f48a96faf53c', 'request_id': 'req-43f44072-e501-4ce9-8252-be4b618d6e78'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:15.054149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e046e84c087f4988b940f48a96faf53c', 'request_id': 'req-43f44072-e501-4ce9-8252-be4b618d6e78'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:15.064654 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:15.076004 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] The token cache is being invalidate because project e046e84c087f4988b940f48a96faf53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92084) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:15.076284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:15.076958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-43f44072-e501-4ce9-8252-be4b618d6e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:15.078813 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1425/2853] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:14 2022] DELETE /identity/v3/projects/e046e84c087f4988b940f48a96faf53c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:15.086288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.086852 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.086959 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.107907 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:15.109979 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:15.170163 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1429/2854] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:15.234959 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.236042 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.236275 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.257054 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:15.258428 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:15.267983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1426/2855] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:15.272554 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.283178 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df6ae9-740a-492b-9529-82f2a20765e6 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.287933 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.288543 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.288821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.289095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] PATH_INFO: `/v3/domains` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.289793 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92083) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:15.290097 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.291735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.305783 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1430/2856] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:15.317353 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.330122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5281-bf65-4ec6-a4d8-7fa265215ba6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.336077 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.336720 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.337008 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.337288 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.338002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.340171 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.467671 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1427/2857] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:15 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:15.473047 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.479659 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:15.486141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a34bc-8869-4f0f-9b40-936b4a227b56 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.545126 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.545802 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.546052 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.546383 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.547066 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.549011 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.763009 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1431/2858] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:15 2022] POST /identity/v3/users => generated 295 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:15.768179 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.779041 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3a7ca-0a38-4687-8f81-1477855c8a15 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.784719 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.785405 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.785726 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.786071 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.786905 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.788667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.800284 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1428/2859] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:15.804805 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.816575 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd49d83-b38b-4a3f-b43b-6c1d26a562d2 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.821473 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.822101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.822412 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.822701 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.823422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.825038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c7fd-182a-4472-a52a-63bd17351194 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.837101 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1432/2860] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:15.845142 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.855913 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542cc09-a557-42ad-a37d-4265eaa0c170 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.860885 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.861555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] REQUEST_METHOD: `PUT` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.861843 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.862126 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] PATH_INFO: `/v3/projects/09dedef0831f4176a315f130608c69c8/users/632dc82b8d2e42579e9bf9019d6d879f/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.877713 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dedef0831f4176a315f130608c69c8, user_id=632dc82b8d2e42579e9bf9019d6d879f, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.880638 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.994421 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1429/2861] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:15 2022] PUT /identity/v3/projects/09dedef0831f4176a315f130608c69c8/users/632dc82b8d2e42579e9bf9019d6d879f/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:16.005735 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.006137 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.006450 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.036690 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] MFA Rules not processed for user `632dc82b8d2e42579e9bf9019d6d879f`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:16.037808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:16.112062 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1433/2862] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:16.116653 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.183299 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340e73-12c9-4523-b1f7-fd943cd51867 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.189534 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.190151 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.190474 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.190860 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.191437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.193338 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.290315 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1430/2863] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:16 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:16.294512 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.305563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590382-70f6-4565-bc54-d8025ca8f99e None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.366598 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.367228 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.367518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.367801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] PATH_INFO: `/v3/users` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.368487 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.370521 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.472416 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1434/2864] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:16 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:16.477615 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.488571 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af10e-6ac0-4ba6-9563-39cb198e3d2a admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.494540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.494540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.494540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.494540 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.495224 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.496927 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.508904 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1431/2865] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:16.513266 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.519230 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:16.527609 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2d8b0-ea52-42fb-a608-c6c04262652a admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.533349 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.533756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.534038 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.534319 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] PATH_INFO: `/v3/projects/8106b3adb4424f479db727ef84bf63b2/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.551644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8106b3adb4424f479db727ef84bf63b2, user_id=ca9f73c76fdf4edeb0e7d54bdd6ca7e7, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.554570 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.626376 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1435/2866] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:16 2022] PUT /identity/v3/projects/8106b3adb4424f479db727ef84bf63b2/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:16.630496 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.641689 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e06c6-7570-4fd9-b242-3b62fd10beb2 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.701202 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.701828 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.702122 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.702522 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.703841 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.706783 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5803e-1471-4695-a441-09eb11046f79 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.717465 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1432/2867] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:16.721711 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.732479 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8dfb-ab40-4390-a650-8d6c6a9eee9e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.738792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.739475 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.739762 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.740072 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] PATH_INFO: `/v3/projects/8106b3adb4424f479db727ef84bf63b2/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7/roles/8fa73b7838ff4d4588a6fe9867cae064` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.742111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8106b3adb4424f479db727ef84bf63b2, user_id=ca9f73c76fdf4edeb0e7d54bdd6ca7e7, role_id=8fa73b7838ff4d4588a6fe9867cae064)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.744816 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.757281 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1436/2868] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:16 2022] PUT /identity/v3/projects/8106b3adb4424f479db727ef84bf63b2/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7/roles/8fa73b7838ff4d4588a6fe9867cae064 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:16.769037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.769037 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.769692 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.777541 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:16.798436 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] MFA Rules not processed for user `ca9f73c76fdf4edeb0e7d54bdd6ca7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:16.802442 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:16.876193 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1433/2869] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:16.886954 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:16.888672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:16.950033 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ef698-7dfe-43ba-bf15-66b8bf3869ab admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:16.956344 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:16.957022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:16.957562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:16.957562 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:16.967603 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:16.969560 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c66f5-b0aa-4800-8fbb-361be427ce92 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:16.979563 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1437/2870] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:17.124214 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:17.192728 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11855628-e30a-42dc-8a1f-66445dc585e1 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:17.197611 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:17.198259 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:17.198573 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:17.198854 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] PATH_INFO: `/v3/users/632dc82b8d2e42579e9bf9019d6d879f` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:17.200043 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=632dc82b8d2e42579e9bf9019d6d879f)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:17.201985 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:17.387628 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632dc82b8d2e42579e9bf9019d6d879f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:17.388290 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632dc82b8d2e42579e9bf9019d6d879f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:17.388932 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632dc82b8d2e42579e9bf9019d6d879f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:17.402745 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632dc82b8d2e42579e9bf9019d6d879f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:17.488602 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632dc82b8d2e42579e9bf9019d6d879f'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:17.502832 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1434/2871] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:17 2022] DELETE /identity/v3/users/632dc82b8d2e42579e9bf9019d6d879f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:17.507727 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:17.508300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:17.508409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:17.529672 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:17.532722 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:17.595536 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1438/2872] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:17.605513 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:17.606996 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:17.618464 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:17.682372 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80deaee-2c39-4630-a1ec-0ab22be23226 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:17.686536 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:17.687265 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:17.687543 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:17.687814 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:17.697824 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:17.699305 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:17.711070 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1435/2873] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:18.140494 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:18.142015 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:18.154821 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598dfdf-4167-48ac-b935-a3c12d66bfef None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:18.160002 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:18.160654 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:18.160936 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:18.161211 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] PATH_INFO: `/v3/projects/09dedef0831f4176a315f130608c69c8` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:18.162385 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09dedef0831f4176a315f130608c69c8)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:18.164322 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:18.266445 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09dedef0831f4176a315f130608c69c8', 'request_id': 'req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.272913 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09dedef0831f4176a315f130608c69c8', 'request_id': 'req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.278513 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09dedef0831f4176a315f130608c69c8', 'request_id': 'req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.280140 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:18.355566 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] The token cache is being invalidate because project 09dedef0831f4176a315f130608c69c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:18.355863 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.356492 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.358022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1439/2874] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:18 2022] DELETE /identity/v3/projects/09dedef0831f4176a315f130608c69c8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:18.362727 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:18.372511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e788c6-d594-41b0-aa20-a7251addcdeb service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:18.431087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:18.431656 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:18.431942 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:18.432222 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] PATH_INFO: `/v3/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:18.433489 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9f73c76fdf4edeb0e7d54bdd6ca7e7)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:18.435555 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:18.639267 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9f73c76fdf4edeb0e7d54bdd6ca7e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.640528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9f73c76fdf4edeb0e7d54bdd6ca7e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.641231 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9f73c76fdf4edeb0e7d54bdd6ca7e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.653026 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9f73c76fdf4edeb0e7d54bdd6ca7e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.733759 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-11975048-ab96-4f62-a607-25f9ed71b9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9f73c76fdf4edeb0e7d54bdd6ca7e7'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:18.747089 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1436/2875] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:18 2022] DELETE /identity/v3/users/ca9f73c76fdf4edeb0e7d54bdd6ca7e7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:19.202051 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:19.204502 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:19.219082 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d2ca-3e94-4c8a-9c7f-a2bc47d12c74 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:19.275406 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:19.275970 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.276253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.276535 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] PATH_INFO: `/v3/projects/8106b3adb4424f479db727ef84bf63b2` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.277680 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8106b3adb4424f479db727ef84bf63b2)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:19.279613 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:19.446678 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8106b3adb4424f479db727ef84bf63b2', 'request_id': 'req-81fc9e10-7abc-4921-9550-f71e297120ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:19.453446 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8106b3adb4424f479db727ef84bf63b2', 'request_id': 'req-81fc9e10-7abc-4921-9550-f71e297120ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:19.460422 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8106b3adb4424f479db727ef84bf63b2', 'request_id': 'req-81fc9e10-7abc-4921-9550-f71e297120ed'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:19.583983 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] The token cache is being invalidate because project 8106b3adb4424f479db727ef84bf63b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:19.584233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:19.584815 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-81fc9e10-7abc-4921-9550-f71e297120ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:19.586333 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1440/2876] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:19 2022] DELETE /identity/v3/projects/8106b3adb4424f479db727ef84bf63b2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:19.592597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.592597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.592597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.612435 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:19.613641 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:19.639587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:19.677528 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1437/2877] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:19.687164 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.687556 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.687866 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.708508 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:19.709732 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:19.718111 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1441/2878] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:19.722520 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:19.732393 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd8ef5-a87d-4932-847a-fbfbc5a8a6a6 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:19.738339 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:19.738990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.739298 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.739579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.740316 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:19.740627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:19.742234 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:19.755568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1438/2879] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:19.760054 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:19.770310 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d0ab0-3da8-481a-a644-86fe3c5b997c None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:19.775179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:19.775801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.776086 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.776409 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.777125 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:19.779081 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:19.865831 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1442/2880] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:19 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:19.875012 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:19.881087 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf834a4-1f1c-4784-a9d2-1359f14aa203 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:19.938098 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:19.938743 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:19.939029 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:19.939308 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:19.939983 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:19.941884 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:20.035437 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1439/2881] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:19 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:15:20.039664 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:20.050830 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9138b-ee9d-4c0f-b353-dfde46331ba0 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:20.055403 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:20.056006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:20.056288 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:20.056564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:20.057253 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:20.058865 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:20.069497 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1443/2882] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:20.077281 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:20.087749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf1b9-4414-4400-b973-dcba04f2acba admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:20.092951 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:20.093706 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:20.093990 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:20.094270 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:20.095033 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:20.096681 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb6e80-7590-4347-85e1-3a789103e00e admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:20.107347 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1440/2883] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:20.111917 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:20.121792 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48ef8-c2c1-43ce-90bf-f675386bc6a6 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:20.127373 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:20.128031 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:20.128311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:20.128586 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] PATH_INFO: `/v3/projects/b2432c71148e42fab47958d3ac127875/users/b77ecf2ce13a4a89928f8cb3ad16bd30/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:20.143112 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2432c71148e42fab47958d3ac127875, user_id=b77ecf2ce13a4a89928f8cb3ad16bd30, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:20.145757 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:20.243006 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1444/2884] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:20 2022] PUT /identity/v3/projects/b2432c71148e42fab47958d3ac127875/users/b77ecf2ce13a4a89928f8cb3ad16bd30/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:20.253685 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:20.254003 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:20.254286 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:20.283581 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] MFA Rules not processed for user `b77ecf2ce13a4a89928f8cb3ad16bd30`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:20.284801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:20.354791 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1441/2885] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:20.366501 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:20.368203 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:20.390519 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:20.450022 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccf4c8-c09e-4a2d-8a9e-dab44935911e admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:20.454728 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e8966ba17c04e1782a6ed9fb4e59254', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:20.455354 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:20.455639 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:20.455916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:20.464918 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:20.466633 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dd670-d021-45aa-86d4-1f8a7be6b6f6 service glance] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:20.476768 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1445/2886] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:30.351712 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:30.487489 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=27 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:37.792113 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:37.861764 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8fbe1-019e-43b2-8f68-196e13ed5f8b None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:37.866297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:37.866961 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:37.867345 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:37.867633 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] PATH_INFO: `/v3/users/b77ecf2ce13a4a89928f8cb3ad16bd30` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:37.868801 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77ecf2ce13a4a89928f8cb3ad16bd30)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:37.870762 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:38.599646 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77ecf2ce13a4a89928f8cb3ad16bd30'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:38.600309 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77ecf2ce13a4a89928f8cb3ad16bd30'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:38.600924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77ecf2ce13a4a89928f8cb3ad16bd30'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:38.612644 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77ecf2ce13a4a89928f8cb3ad16bd30'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:38.773157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77ecf2ce13a4a89928f8cb3ad16bd30'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:38.787568 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1442/2887] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:37 2022] DELETE /identity/v3/users/b77ecf2ce13a4a89928f8cb3ad16bd30 => generated 0 bytes in 996 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:38.792362 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:38.792889 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:38.793000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:38.812055 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:38.813300 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:38.873258 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1446/2888] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:38.883268 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:38.884702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:38.944367 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f8f6-bfcd-4013-b4c1-dacc67e5c0fc admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:38.947988 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:38.948608 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:38.948895 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:38.949173 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:38.958888 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:38.960321 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426f3b5-118c-4cba-9114-9b90b84dd3ff service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:38.969933 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1443/2889] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:39.443759 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:39.445277 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:39.458141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08c07-c3fb-4cf2-a6dd-a01b326bf370 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:39.462850 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:39.463518 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:39.463817 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:39.464095 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] PATH_INFO: `/v3/projects/b2432c71148e42fab47958d3ac127875` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:39.465660 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2432c71148e42fab47958d3ac127875)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:39.467629 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:39.597855 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2432c71148e42fab47958d3ac127875', 'request_id': 'req-8e94e013-da77-444b-8ed6-36a529dcdbd8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:39.604064 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2432c71148e42fab47958d3ac127875', 'request_id': 'req-8e94e013-da77-444b-8ed6-36a529dcdbd8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:39.609565 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2432c71148e42fab47958d3ac127875', 'request_id': 'req-8e94e013-da77-444b-8ed6-36a529dcdbd8'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:39.698259 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] The token cache is being invalidate because project b2432c71148e42fab47958d3ac127875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:39.698582 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:39.699244 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-8e94e013-da77-444b-8ed6-36a529dcdbd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:39.700956 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1447/2890] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:39 2022] DELETE /identity/v3/projects/b2432c71148e42fab47958d3ac127875 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:39.706997 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:39.707293 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:39.707584 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:39.727559 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:39.728766 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:39.746511 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:39.791966 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1444/2891] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:39.801275 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:39.801801 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:39.801916 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:39.821928 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:39.823223 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:39.832141 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1448/2892] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:39.836334 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:39.846896 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6ce68-b2d4-4d37-8965-081598e89c80 None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:39.851711 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:39.852295 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:39.852582 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:39.852864 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:39.853566 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92084) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:39.853881 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:39.855546 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:39.867189 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1445/2893] 10.222.0.85 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:39.871836 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:39.881644 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9951c-3abd-4158-908f-6a85502db8be None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:39.886379 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:39.887000 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:39.887284 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:39.887564 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] PATH_INFO: `/v3/projects` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:39.888227 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:39.890117 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:39.980019 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1449/2894] 10.222.0.85 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:39 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:39.984510 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:39.995099 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4554e88-fa76-4f57-a0d1-eec4688abae8 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:40.052893 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:40.053579 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.053867 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.054152 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.054857 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:40.056776 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:40.144818 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1446/2895] 10.222.0.85 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:39 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:40.149053 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:40.159684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34cd1e-b353-41de-ba9e-72d080d7fe63 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:40.164408 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:40.165020 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.165311 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.165641 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] PATH_INFO: `/v3/roles` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.166365 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:40.167973 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:40.178780 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1450/2896] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:40.182844 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:40.193515 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717395-e9e2-4c70-888c-791c81d69dc3 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:40.198238 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:40.198883 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.199169 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.199449 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.200148 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:40.201749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acddbc-f3ee-4ae8-94a9-5b7217225e89 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:40.212569 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1447/2897] 10.222.0.85 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:40.216937 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:40.227179 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4619b4c-e44e-4c49-b843-9255e33ff309 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:40.231823 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:40.232504 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] REQUEST_METHOD: `PUT` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.232794 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.233077 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] PATH_INFO: `/v3/projects/be5053060d5e40ce965ba58b59ae47aa/users/0e8c0dfd07d54a92b89014635b0b65ec/roles/ec41984b84ff479d8a0e7ad9405aa8b1` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.248257 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5053060d5e40ce965ba58b59ae47aa, user_id=0e8c0dfd07d54a92b89014635b0b65ec, role_id=ec41984b84ff479d8a0e7ad9405aa8b1)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:40.251756 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:40.402154 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1451/2898] 10.222.0.85 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:40 2022] PUT /identity/v3/projects/be5053060d5e40ce965ba58b59ae47aa/users/0e8c0dfd07d54a92b89014635b0b65ec/roles/ec41984b84ff479d8a0e7ad9405aa8b1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:40.413135 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] REQUEST_METHOD: `POST` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.413429 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.413710 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.442504 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.auth.core [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] MFA Rules not processed for user `0e8c0dfd07d54a92b89014635b0b65ec`. Rule list: `[]` (Enabled: `True`). {{(pid=92084) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:40.443781 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:40.508749 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1448/2899] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:40.520076 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:40.521771 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:40.543867 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:40.579790 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da997f-6a62-4a00-9dbd-cfbfe56bd373 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:40.583630 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4573990adc20477ab369cd717aaf0679', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:40.588822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:40.588822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:40.588822 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:40.593685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:40.595886 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e73fbb-cc8f-4522-be80-5d0d47799340 service cinder] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:40.604685 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1452/2900] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:44.622532 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:44.642002 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:44.685647 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9675d9-3233-4488-b88a-2a861e71c8ce None None] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:44.690619 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:44.691297 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:44.691605 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:44.691924 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] PATH_INFO: `/v3/users/0e8c0dfd07d54a92b89014635b0b65ec` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:44.693111 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8c0dfd07d54a92b89014635b0b65ec)` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:44.695149 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:44.895667 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8c0dfd07d54a92b89014635b0b65ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:44.896327 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8c0dfd07d54a92b89014635b0b65ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:44.896958 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8c0dfd07d54a92b89014635b0b65ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:44.909953 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8c0dfd07d54a92b89014635b0b65ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:44.999627 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.notifications [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8c0dfd07d54a92b89014635b0b65ec'} {{(pid=92084) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.013444 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1449/2901] 10.222.0.85 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:44 2022] DELETE /identity/v3/users/0e8c0dfd07d54a92b89014635b0b65ec => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:45.018153 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] REQUEST_METHOD: `POST` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:45.018684 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:45.018808 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:45.038592 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.auth.core [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] MFA Rules not processed for user `5580a3ecfd9d47fb8a889209b62e0089`. Rule list: `[]` (Enabled: `True`). {{(pid=92083) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:45.039810 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:45.098646 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1453/2902] 10.222.0.85 () {58 vars in 958 bytes} [Thu Jul 28 18:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:45.108246 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:45.109733 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:45.167301 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120060d-b75e-482b-af3a-24537114b877 admin admin] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:45.170984 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ebf672346841c8addaa849fc11a052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:45.171597 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:45.171879 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:45.172257 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:45.181593 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:45.183066 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:45.192912 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1450/2903] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:45.605588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:45.605588 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:45.620668 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088ce5-6b27-40f8-be34-8a6fe0ec1502 None None] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:45.627262 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5580a3ecfd9d47fb8a889209b62e0089', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eaffcb498d9c405ca0d18c97b43881f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:45.627952 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:45.628233 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:45.628532 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] PATH_INFO: `/v3/projects/be5053060d5e40ce965ba58b59ae47aa` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:45.629697 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5053060d5e40ce965ba58b59ae47aa)` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:45.631640 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:45.727925 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5053060d5e40ce965ba58b59ae47aa', 'request_id': 'req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.734039 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5053060d5e40ce965ba58b59ae47aa', 'request_id': 'req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.740282 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5053060d5e40ce965ba58b59ae47aa', 'request_id': 'req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949'} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.807550 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] The token cache is being invalidate because project be5053060d5e40ce965ba58b59ae47aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92083) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:45.808003 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.808731 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.notifications [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92083) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:45.810468 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1454/2904] 10.222.0.85 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:45 2022] DELETE /identity/v3/projects/be5053060d5e40ce965ba58b59ae47aa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:55.188875 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:55.804210 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG dbcounter [-] [92083] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92083) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:01.041209 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] Authenticating user token {{(pid=92084) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:01.042665 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:01.113702 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498cfef-dff2-487e-8f4d-be7e3a5e0e20 service neutron] Validating token access rules against request {{(pid=92084) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:01.173940 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92084) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:01.174587 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] REQUEST_METHOD: `GET` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:01.174876 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] SCRIPT_NAME: `/identity` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:01.175157 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92084) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:01.177133 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.fernet_utils [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92084) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:01.188457 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:01.190585 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c84d-ec36-4361-8984-7f7be3c4a3ca service placement] RBAC: Authorization granted {{(pid=92084) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:01.200413 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: [pid: 92084|app: 0|req: 1451/2905] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:11.196132 dv-cinder-837543-7-iscsi devstack@keystone.service[92084]: DEBUG dbcounter [-] [92084] Writing DB stats keystone:SELECT=58 {{(pid=92084) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:21:02.004145 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Authenticating user token {{(pid=92083) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:21:02.005748 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.fernet_utils [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92083) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:21:02.095034 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa6061-1f2e-4cc0-bfe3-f5ec8252f949 admin admin] Validating token access rules against request {{(pid=92083) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:21:02.100456 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c8ff3469974d15a2a3ec8d7af9fe07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b133df81b374a9e944d96078dfd1d55', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92083) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:21:02.101109 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] REQUEST_METHOD: `GET` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:21:02.101398 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] SCRIPT_NAME: `/identity` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:21:02.101681 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92083) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:21:02.111599 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:21:02.113246 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46575-5da0-41bf-a299-92d1a6bb79c3 service placement] RBAC: Authorization granted {{(pid=92083) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:21:02.124679 dv-cinder-837543-7-iscsi devstack@keystone.service[92083]: [pid: 92083|app: 0|req: 1455/2906] 10.222.0.85 () {60 vars in 1365 bytes} [Thu Jul 28 18:21:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)